Sep 10 20:56:44.361549 ip-10-0-135-142 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 20:56:44.361558 ip-10-0-135-142 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 20:56:44.361565 ip-10-0-135-142 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 20:56:44.361834 ip-10-0-135-142 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 20:56:53.622128 ip-10-0-135-142 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 20:56:53.622142 ip-10-0-135-142 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 4b2e425bb369461297ae3c8df7dc1643 --
Sep 10 20:58:26.215404 ip-10-0-135-142 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 20:58:26.642489 ip-10-0-135-142 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 20:58:26.642489 ip-10-0-135-142 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 20:58:26.642489 ip-10-0-135-142 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 20:58:26.642489 ip-10-0-135-142 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 20:58:26.642489 ip-10-0-135-142 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 20:58:26.644550 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.644467    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 20:58:26.647885 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647870    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:26.647885 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647886    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647890    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647893    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647896    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647900    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647903    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647906    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647909    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647912    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647914    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647917    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647920    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647923    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647925    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647932    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647935    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647938    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647941    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647945    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647947    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:26.647947 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647950    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647953    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647956    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647959    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647962    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647964    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647969    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647973    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647976    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647979    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647982    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647985    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647987    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647990    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647993    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647996    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.647999    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648001    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648004    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:26.648446 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648006    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648010    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648013    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648017    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648020    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648023    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648025    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648028    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648030    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648034    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648036    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648039    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648041    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648044    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648047    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648052    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648067    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648070    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648072    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648075    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:26.648956 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648078    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648081    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648083    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648086    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648088    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648091    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648093    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648096    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648098    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648101    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648104    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648106    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648110    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648112    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648115    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648117    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648120    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648123    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648126    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648128    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:26.649551 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648131    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648133    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648140    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648143    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648145    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648148    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648150    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648154    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648157    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648159    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648162    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648164    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648167    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648170    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648173    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648175    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648178    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648180    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648183    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648870    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:26.650177 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648877    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648881    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648886    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648890    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648894    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648897    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648900    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648903    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648906    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648910    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648913    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648916    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648919    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648922    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648925    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648928    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648930    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648933    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648936    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648939    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:26.650862 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648941    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648944    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648947    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648949    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648952    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648955    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648957    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648960    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648962    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648965    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648968    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648971    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648974    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648977    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648981    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648986    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648989    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648992    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648995    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:26.651383 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.648997    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649000    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649003    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649006    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649008    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649011    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649014    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649016    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649019    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649021    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649024    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649027    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649029    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649033    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649036    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649038    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649041    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649043    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649046    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649048    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:26.651889 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649051    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649053    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649071    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649074    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649077    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649079    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649082    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649084    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649087    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649090    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649093    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649095    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649098    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649101    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649103    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649106    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649109    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649112    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649115    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649117    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:26.652419 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649120    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649123    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649126    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649128    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649131    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649133    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649136    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649139    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649141    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649144    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649146    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649149    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649151    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649153    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649157    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649161    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649164    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649166    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.649169    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:26.652916 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651082    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651094    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651098    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651101    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651104    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651107    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651109    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651112    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651115    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651118    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651121    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651125    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651128    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651130    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651133    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651136    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651139    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651141    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651144    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:26.653400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651147    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651149    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651152    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651155    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651157    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651164    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651166    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651169    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651172    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651175    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651177    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651180    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651183    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651186    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651189    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651192    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651195    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651197    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651200    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651203    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:26.653877 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651207    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651211    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651214    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651217    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651220    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651224    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651226    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651229    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651232    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651234    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651237    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651240    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651242    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651245    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651248    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651250    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651253    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651256    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651258    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651261    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:26.654387 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651264    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651267    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651270    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651272    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651275    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651277    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651280    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651284    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651286    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651289    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651291    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651294    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651297    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651299    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651301    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651304    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651307    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651309    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651312    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651314    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:26.654869 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651317    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651319    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651322    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651324    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651327    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651329    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651332    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651336    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651340    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651343    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651345    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651348    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651350    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651354    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651356    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651359    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651363    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651367    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651370    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.651373    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:26.655363 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651453    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651501    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651510    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651514    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651519    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651522    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651527    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651532    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651535    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651538    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651542    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651545    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651548    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651551    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651554    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651557    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651561    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651565    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651569    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651572    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651575    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651578    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651582    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651586    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 20:58:26.655847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651589    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651592    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651596    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651599    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651602    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651605    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651609    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651613    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651617    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651620    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651623    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651626    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651629    2580 flags.go:64] FLAG: --enable-server="true"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651632    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651637    2580 flags.go:64] FLAG: --event-burst="100"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651640    2580 flags.go:64] FLAG: --event-qps="50"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651643    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651646    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651650    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651653    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651656    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651660    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651663    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651666    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651669    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 20:58:26.656441 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651672    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651675    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651678    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651681    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651684    2580 flags.go:64] FLAG: --feature-gates=""
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651692    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651695    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651699    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651702    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651705    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651708    2580 flags.go:64] FLAG: --help="false"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651711    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-135-142.ec2.internal"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651714    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651718    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651721    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651724    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651728    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651731    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651734    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651737    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651740    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651743    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651746    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651749    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 10 20:58:26.657048 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651752    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651754    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651757    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651760    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651763    2580 flags.go:64] FLAG: --lock-file=""
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651766    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651769    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651772    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651777    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651781    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651784    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651786    2580 flags.go:64] FLAG: --logging-format="text"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651789    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651793    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651796    2580 flags.go:64] FLAG: --manifest-url=""
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651799    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651806    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651809    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651813    2580 flags.go:64] FLAG: --max-pods="110"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651817    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651820    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651823    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651826    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651829    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651832    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 20:58:26.657641 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651835    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651842    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651846    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651849    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651852    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651855    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651860    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651863    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651866    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651869    2580 flags.go:64] FLAG: --port="10250"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651872    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651876    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b55ea616c8372b2e"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651879    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651882    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651885    2580 flags.go:64] FLAG: --register-node="true"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651887    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651892    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651896    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651899    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651902    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651906    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651909    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651912    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651915    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651920    2580 flags.go:64] FLAG: --runonce="false"
Sep 10 20:58:26.658285 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651923    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651926    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651929    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651932    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651935    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651938    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651941    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651945    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651948    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651951    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651954    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651957    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651960    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651963    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651966    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651972    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651975    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651978    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651982    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651984    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651987    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651990    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651993    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.651996    2580 flags.go:64] FLAG: --v="2"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.652000    2580 flags.go:64] FLAG: --version="false"
Sep 10 20:58:26.658935 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.652005    2580 flags.go:64] FLAG: --vmodule=""
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.652009    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.652012    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652136    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652140    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652143    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652148    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652150    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652153    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652156    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652159    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652162    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652165    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652168    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652170    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652173    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652176    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652179    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652181    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652184    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:26.659564 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652187    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652189    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652192    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652195    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652197    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652200    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652202    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652205    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652207    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652210    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652213    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652215    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652218    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652222    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652226    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652229    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652231    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652234    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652238    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652241    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:26.660147 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652243    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652246    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652249    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652252    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652254    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652257    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652259    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652262    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652265    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652268    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652270    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652273    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652276    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652278    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652281    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652284    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652286    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652289    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652293    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652297    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:26.660766 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652300    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652303    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652306    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652308    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652312    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652315    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652321    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652323    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652326    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652329    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652333    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652335    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652338    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652341    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652343    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652346    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652349    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652352    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652354    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:26.661314 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652357    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652359    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652362    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652365    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652367    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652370    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652373    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652376    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652378    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652381    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652384    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652389    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652395    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652399    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652402    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652406    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652408    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652411    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652414    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652418    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:26.661795 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652421    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652424    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.652427    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.653126    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.660938    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.660952    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.660970    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.660976    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661031    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661036    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661039    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661043    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661047    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661051    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:26.662310 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661072    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661076    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661079    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661082    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661085    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661088    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661091    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661094    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661097    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661100    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661102    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661105    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661108    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661111    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661113    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661116    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661118    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661121    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661125    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661128    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:26.662663 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661131    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661134    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661136    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661140    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661143    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661145    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661148    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661151    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661153    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661156    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661159    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661161    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661164    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661166    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661169    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661172    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661175    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661178    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661180    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661183    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:26.663175 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661186    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661189    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661192    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661195    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661197    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661200    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661203    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661205    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661208    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661211    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661213    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661216    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661221    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661224    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661227    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661230    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661232    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661235    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661238    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661240    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:26.663647 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661243    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661245    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661248    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661251    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661253    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661256    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661258    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661261    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661264    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661266    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661269    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661271    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661275    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661277    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661280    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661283    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661286    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661288    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661291    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661293    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:26.664227 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661297    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661300    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661303    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661307    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661311    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661314    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661317    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661320    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661322    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661325    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661328    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661331    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661334    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.661339    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 20:58:26.664708 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661448    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661453    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661456    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661459    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661462    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661465    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661468    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661471    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661474    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661478    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661481    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661484    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661487    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661489    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661492    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661494    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661497    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661500    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661502    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:26.665039 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661505    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661507    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661510    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661513    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661517    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661520    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661522    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661525    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661527    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661530    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661533    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661535    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661538    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661541    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661544    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661546    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661549    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661552    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661555    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661557    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:26.665518 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661560    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661562    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661565    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661567    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661571    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661575    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661578    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661581    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661583    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661586    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661588    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661591    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661593    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661596    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661599    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661601    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661604    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661607    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661610    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661613    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:26.665990 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661615    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661618    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661620    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661623    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661626    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661628    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661631    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661634    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661636    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661639    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661642    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661645    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661649    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661652    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661655    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661658    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661661    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661664    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661667    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:26.666487 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661670    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661672    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661675    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661678    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661680    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661683    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661686    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661688    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661691    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661694    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661697    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661700    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661702    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661705    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661708    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661711    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661714    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661717    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661720    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661723    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:26.666979 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:26.661726    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:26.667476 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.661731    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 20:58:26.667476 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.661859    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 20:58:26.667476 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.664250    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 20:58:26.667476 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.665349    2580 server.go:1019] "Starting client certificate rotation"
Sep 10 20:58:26.667476 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.665448    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 20:58:26.667476 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.665486    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 20:58:26.668269 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.668256    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 20:58:26.669463 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.669446    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 20:58:26.680101 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.680085    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 10 20:58:26.683528 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.683510    2580 log.go:25] "Validated CRI v1 image API"
Sep 10 20:58:26.684509 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.684488    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 20:58:26.688386 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.688352    2580 fs.go:135] Filesystem UUIDs: map[18969ddf-31f5-4e12-b6f5-b488f7beb6ad:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9658242e-a9cd-4a88-90aa-0afd370a17fd:/dev/nvme0n1p4]
Sep 10 20:58:26.688470 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.688387    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 20:58:26.694337 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.694213    2580 manager.go:216] Machine: {Timestamp:2026-09-10 20:58:26.693704922 +0000 UTC m=+0.373203763 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100775 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c0bff22e22f533a1120c320160dd2 SystemUUID:ec2c0bff-22e2-2f53-3a11-20c320160dd2 BootID:4b2e425b-b369-4612-97ae-3c8df7dc1643 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7b:47:4c:9b:5b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7b:47:4c:9b:5b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:9a:94:cb:aa:df Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 20:58:26.694337 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.694323    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 20:58:26.694475 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.694403    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 20:58:26.698250 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.698221    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 20:58:26.698387 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.698253    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-142.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 20:58:26.699043 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.699032    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 20:58:26.699090 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.699046    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 20:58:26.699090 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.699064    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 20:58:26.699828 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.699818    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 20:58:26.701066 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.701047    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 20:58:26.705701 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.705686    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 20:58:26.705830 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.705820    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 20:58:26.705875 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.705851    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 20:58:26.705875 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.705860    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 20:58:26.705875 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.705872    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 20:58:26.705977 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.705881    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 10 20:58:26.705977 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.705890    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 20:58:26.706405 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.706274    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 20:58:26.706469 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.706413    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 20:58:26.707959 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.707939    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 20:58:26.710578 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.710546    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 20:58:26.718647 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:26.718623    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-142.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 20:58:26.718647 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:26.718625    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 20:58:26.757302 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.757277    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-74r4g"
Sep 10 20:58:26.765375 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.765355    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-74r4g"
Sep 10 20:58:26.832829 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.832808    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 20:58:26.832904 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.832841    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 20:58:26.835666 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835650    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 20:58:26.835709 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835679    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 20:58:26.835709 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835685    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 20:58:26.835709 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835690    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 20:58:26.835709 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835699    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 20:58:26.835709 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835705    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 20:58:26.835843 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835720    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 20:58:26.835843 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835726    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 20:58:26.835843 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835733    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 20:58:26.835843 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835739    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 20:58:26.835843 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835747    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 20:58:26.835843 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835752    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 20:58:26.835843 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835781    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 20:58:26.835843 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.835789    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 20:58:26.839685 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.839671    2580 server.go:1268] "Started kubelet"
Sep 10 20:58:26.839796 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.839773    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 20:58:26.839847 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.839800    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 20:58:26.839888 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.839874    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 20:58:26.840479 ip-10-0-135-142 systemd[1]: Started Kubernetes Kubelet.
Sep 10 20:58:26.841379 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.841289    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 20:58:26.842448 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.842431    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 20:58:26.844742 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:26.844723    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 20:58:26.852301 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.852281    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 20:58:26.852301 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.852296    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 20:58:26.852441 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:26.852423    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:26.853098 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.853080    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 20:58:26.853192 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.853110    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 20:58:26.853192 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.853147    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 20:58:26.854926 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.854907    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:26.855581 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.855555    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 20:58:26.855581 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.855573    2580 factory.go:55] Registering systemd factory
Sep 10 20:58:26.855711 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.855585    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 10 20:58:26.855895 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.855869    2580 factory.go:153] Registering CRI-O factory
Sep 10 20:58:26.855895 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.855890    2580 factory.go:223] Registration of the crio container factory successfully
Sep 10 20:58:26.856010 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.855910    2580 factory.go:103] Registering Raw factory
Sep 10 20:58:26.856010 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.855926    2580 manager.go:1196] Started watching for new ooms in manager
Sep 10 20:58:26.856140 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.856099    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 20:58:26.856140 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.856109    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 20:58:26.856623 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.856609    2580 manager.go:349] Starting recovery of all containers
Sep 10 20:58:26.858281 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:26.858253    2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-142.ec2.internal\" not found" node="ip-10-0-135-142.ec2.internal"
Sep 10 20:58:26.859942 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.859922    2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-142.ec2.internal" not found
Sep 10 20:58:26.867471 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.867457    2580 manager.go:354] Recovery completed
Sep 10 20:58:26.871427 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.871414    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:26.873076 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.873050    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:26.873140 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.873089    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:26.873140 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.873099    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:26.873636 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.873624    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 20:58:26.873677 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.873636    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 20:58:26.873677 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.873652    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 20:58:26.873740 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.873638    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:26.874368 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.874354    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:26.874444 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.874383    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:26.874444 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.874398    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:26.875836 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.875823    2580 policy_none.go:49] "None policy: Start"
Sep 10 20:58:26.875836 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.875839    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 20:58:26.875929 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.875849    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 20:58:26.877831 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.877821    2580 policy_none.go:47] "Start"
Sep 10 20:58:26.881416 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.881356    2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-142.ec2.internal" not found
Sep 10 20:58:26.921436 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.919291    2580 manager.go:337] "Starting Device Plugin manager"
Sep 10 20:58:26.921436 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:26.919330    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 20:58:26.921436 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.919342    2580 server.go:87] "Starting device plugin registration server"
Sep 10 20:58:26.921436 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.919573    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 20:58:26.921436 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.919584    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 20:58:26.921436 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.919663    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 20:58:26.921436 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.919736    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 20:58:26.921436 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.919745    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 20:58:26.921436 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:26.920280    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 20:58:26.921436 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:26.920328    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:26.939879 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.939862    2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-142.ec2.internal" not found
Sep 10 20:58:26.948317 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.948289    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 20:58:26.949508 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.949485    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 20:58:26.949586 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.949544    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 20:58:26.949586 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.949571    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 20:58:26.949672 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:26.949609    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 20:58:26.952249 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:26.952234    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:27.020230 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.020195    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:27.021215 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.021199    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:27.021289 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.021234    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:27.021289 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.021249    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:27.021289 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.021277    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.030722 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.030704    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.030773 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:27.030726    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-142.ec2.internal\": node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:27.050391 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.050370    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-142.ec2.internal"]
Sep 10 20:58:27.050474 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.050433    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:27.051245 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.051228    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:27.051342 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.051253    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:27.051342 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.051266    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:27.053500 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.053487    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:27.053628 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.053612    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.053688 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.053649    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:27.054224 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.054208    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:27.054297 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.054212    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:27.054297 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.054264    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:27.054297 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.054280    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:27.054297 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.054238    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:27.054448 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.054300    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:27.056485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.056467    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.056578 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.056492    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:27.057108 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.057089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed1b1f179b2e10e70b3c97f4374d819e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal\" (UID: \"ed1b1f179b2e10e70b3c97f4374d819e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.057196 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.057120    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:27.057196 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.057137    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:27.057196 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.057150    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:27.057196 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.057168    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed1b1f179b2e10e70b3c97f4374d819e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal\" (UID: \"ed1b1f179b2e10e70b3c97f4374d819e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.058743 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:27.058727    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:27.081554 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:27.081535    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-142.ec2.internal\" not found" node="ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.085664 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:27.085648    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-142.ec2.internal\" not found" node="ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.158034 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.157965    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed1b1f179b2e10e70b3c97f4374d819e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal\" (UID: \"ed1b1f179b2e10e70b3c97f4374d819e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.158034 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.157995    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed1b1f179b2e10e70b3c97f4374d819e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal\" (UID: \"ed1b1f179b2e10e70b3c97f4374d819e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.158034 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.158017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/281550d9d955d980d569b392e95bf98a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-142.ec2.internal\" (UID: \"281550d9d955d980d569b392e95bf98a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.158253 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.158078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed1b1f179b2e10e70b3c97f4374d819e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal\" (UID: \"ed1b1f179b2e10e70b3c97f4374d819e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.158253 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.158096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed1b1f179b2e10e70b3c97f4374d819e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal\" (UID: \"ed1b1f179b2e10e70b3c97f4374d819e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.159014 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:27.158989    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:27.259128 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:27.259092    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:27.259270 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.259148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/281550d9d955d980d569b392e95bf98a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-142.ec2.internal\" (UID: \"281550d9d955d980d569b392e95bf98a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.259270 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.259210    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/281550d9d955d980d569b392e95bf98a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-142.ec2.internal\" (UID: \"281550d9d955d980d569b392e95bf98a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.360069 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:27.360034    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:27.385071 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.385042    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.388487 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.388464    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:27.460703 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:27.460623    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:27.560996 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:27.560956    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:27.662072 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:27.662028    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:27.665461 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.665425    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 20:58:27.665630 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.665610    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 20:58:27.665690 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.665639    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 20:58:27.763178 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:27.763080    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:27.786894 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.786863    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:27.836364 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.836270    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 20:53:26 +0000 UTC" deadline="2028-04-02 15:33:17.016947413 +0000 UTC"
Sep 10 20:58:27.836364 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:27.836360    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13674h34m49.180592005s"
Sep 10 20:58:27.863738 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:27.863704    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:27.964536 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:27.964505    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:27.995230 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:27.995199    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod281550d9d955d980d569b392e95bf98a.slice/crio-58cb81deb8dcd21259af3e0ee34050514fef47f9cceb42d647e890afef37e1ba WatchSource:0}: Error finding container 58cb81deb8dcd21259af3e0ee34050514fef47f9cceb42d647e890afef37e1ba: Status 404 returned error can't find the container with id 58cb81deb8dcd21259af3e0ee34050514fef47f9cceb42d647e890afef37e1ba
Sep 10 20:58:27.998535 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:27.997215    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded1b1f179b2e10e70b3c97f4374d819e.slice/crio-fe25ca304c33f665d80bbb3d5579179ea9cda6a856fd90400d3dc2eda917fccd WatchSource:0}: Error finding container fe25ca304c33f665d80bbb3d5579179ea9cda6a856fd90400d3dc2eda917fccd: Status 404 returned error can't find the container with id fe25ca304c33f665d80bbb3d5579179ea9cda6a856fd90400d3dc2eda917fccd
Sep 10 20:58:28.001101 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.001081    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 20:58:28.064984 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:28.064872    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-142.ec2.internal\" not found"
Sep 10 20:58:28.108234 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.108207    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:28.152943 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.152906    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:28.169206 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.169182    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 20:58:28.169996 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.169983    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal"
Sep 10 20:58:28.183717 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.183698    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 20:58:28.708405 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.708275    2580 apiserver.go:52] "Watching apiserver"
Sep 10 20:58:28.717217 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.717184    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 20:58:28.717705 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.717670    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal","openshift-multus/multus-additional-cni-plugins-8mxtq","openshift-multus/network-metrics-daemon-dkc7w","kube-system/konnectivity-agent-2ch2t","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl","openshift-cluster-node-tuning-operator/tuned-d6fcr","openshift-image-registry/node-ca-4gz27","openshift-multus/multus-h9lm2","openshift-network-diagnostics/network-check-target-pcwdg","openshift-network-operator/iptables-alerter-mxxg4","openshift-ovn-kubernetes/ovnkube-node-2qkd6","kube-system/global-pull-secret-syncer-k7pdz","kube-system/kube-apiserver-proxy-ip-10-0-135-142.ec2.internal","openshift-dns/node-resolver-xhglt"]
Sep 10 20:58:28.720503 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.720480    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4gz27"
Sep 10 20:58:28.723039 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.723014    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.723350 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.723329    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.723620 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.723594    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qkcgh\""
Sep 10 20:58:28.723929 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.723864    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 20:58:28.724213 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.724193    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.726728 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.726127    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.726728 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.726399    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 20:58:28.726728 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.726429    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.726728 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.726607    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:28.726728 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.726624    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 20:58:28.726728 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:28.726670    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:28.727645 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.727405    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 20:58:28.727645 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.727440    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2sptm\""
Sep 10 20:58:28.727645 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.727449    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 20:58:28.728249 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.728229    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2ch2t"
Sep 10 20:58:28.730725 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.730701    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 20:58:28.731025 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.731005    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 20:58:28.731128 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.731064    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.731297 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.731281    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cfp2h\""
Sep 10 20:58:28.733521 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.733453    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.733973 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.733683    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 20:58:28.733973 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.733815    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.735031 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.735005    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.736447 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.736425    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.736727 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.736699    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.737129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.737110    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:28.737446 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:28.737405    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:28.737585 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.737457    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xslb9\""
Sep 10 20:58:28.738905 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.738884    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ddn26\""
Sep 10 20:58:28.741093 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.740598    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xhglt"
Sep 10 20:58:28.743223 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.743199    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.743579 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.743542    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-d7mp8\""
Sep 10 20:58:28.743679 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.743659    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.745182 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.744536    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.747000 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.746972    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 20:58:28.747000 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.746997    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mxxg4"
Sep 10 20:58:28.747897 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.747666    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.747897 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.747714    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6cthj\""
Sep 10 20:58:28.747897 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.747724    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 20:58:28.748118 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.747951    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 20:58:28.748219 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.748191    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.749991 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.749751    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 20:58:28.749991 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.749779    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.749991 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.749881    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4gzfv\""
Sep 10 20:58:28.750563 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.750539    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 20:58:28.750970 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.750952    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 20:58:28.752657 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.752343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:28.752657 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:28.752419    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:28.752657 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.752568    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pzvmp\""
Sep 10 20:58:28.752882 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.752763    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 20:58:28.753023 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.753004    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:28.754388 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.754324    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 20:58:28.768619 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.768591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-var-lib-openvswitch\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.768747 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.768671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-run-openvswitch\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.768747 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.768703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea3e1564-e8e9-47db-a5e3-86002afeaeab-host\") pod \"node-ca-4gz27\" (UID: \"ea3e1564-e8e9-47db-a5e3-86002afeaeab\") " pod="openshift-image-registry/node-ca-4gz27"
Sep 10 20:58:28.768747 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.768730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-system-cni-dir\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.768925 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.768816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5fab6eb-e449-42a1-b535-0728a6a1be03-ovnkube-config\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.768925 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.768861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0819e2f5-b35c-4836-8d5e-939990ad6009-kubelet-config\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:28.768925 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.768920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:28.769082 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.768948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-systemd\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.769082 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.768973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-var-lib-cni-multus\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.769212 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9837b724-5c2c-49df-b071-5e596f11562b-multus-daemon-config\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.769212 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72dzk\" (UniqueName: \"kubernetes.io/projected/9837b724-5c2c-49df-b071-5e596f11562b-kube-api-access-72dzk\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.769212 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769163    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-cni-bin\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.769212 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43499b3a-6698-490b-831a-70b2868648bf-tmp\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.769480 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769210    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.769480 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-run-netns\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.769480 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769288    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-run-multus-certs\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.769480 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-socket-dir\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.769480 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-multus-socket-dir-parent\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.769480 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769369    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-kubelet\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.769480 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769449    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c6gp\" (UniqueName: \"kubernetes.io/projected/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-kube-api-access-4c6gp\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.769837 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769488    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xls6c\" (UniqueName: \"kubernetes.io/projected/ed9cd22c-bd90-473a-9814-eea2735c3cf0-kube-api-access-xls6c\") pod \"node-resolver-xhglt\" (UID: \"ed9cd22c-bd90-473a-9814-eea2735c3cf0\") " pod="openshift-dns/node-resolver-xhglt"
Sep 10 20:58:28.769837 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769541    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba107ada-a3fd-4808-bfad-7105a2dc25f5-konnectivity-ca\") pod \"konnectivity-agent-2ch2t\" (UID: \"ba107ada-a3fd-4808-bfad-7105a2dc25f5\") " pod="kube-system/konnectivity-agent-2ch2t"
Sep 10 20:58:28.769837 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-modprobe-d\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.769837 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-lib-modules\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.769837 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769689    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-hostroot\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.769837 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-etc-kubernetes\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.769837 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl4gb\" (UniqueName: \"kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb\") pod \"network-check-target-pcwdg\" (UID: \"b52aa986-0758-4fd0-a230-498beba6b387\") " pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:28.769837 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0819e2f5-b35c-4836-8d5e-939990ad6009-dbus\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:28.769837 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769801    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ea3e1564-e8e9-47db-a5e3-86002afeaeab-serviceca\") pod \"node-ca-4gz27\" (UID: \"ea3e1564-e8e9-47db-a5e3-86002afeaeab\") " pod="openshift-image-registry/node-ca-4gz27"
Sep 10 20:58:28.769837 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zqm9\" (UniqueName: \"kubernetes.io/projected/43499b3a-6698-490b-831a-70b2868648bf-kube-api-access-5zqm9\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-etc-openvswitch\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769895    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5fab6eb-e449-42a1-b535-0728a6a1be03-env-overrides\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed9cd22c-bd90-473a-9814-eea2735c3cf0-tmp-dir\") pod \"node-resolver-xhglt\" (UID: \"ed9cd22c-bd90-473a-9814-eea2735c3cf0\") " pod="openshift-dns/node-resolver-xhglt"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.769963    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-kubernetes\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-sysctl-conf\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770052    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-host\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-run-ovn-kubernetes\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-sysconfig\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770148    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-sys\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770172    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-var-lib-cni-bin\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-log-socket\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5fab6eb-e449-42a1-b535-0728a6a1be03-ovnkube-script-lib\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-sys-fs\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770285    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-run\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770331    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-cnibin\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.770367 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-cni-netd\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770365    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jbz4\" (UniqueName: \"kubernetes.io/projected/1bb0820e-193e-416f-a950-5eedb01fc196-kube-api-access-4jbz4\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770420    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-sysctl-d\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-var-lib-kubelet\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770466    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/43499b3a-6698-490b-831a-70b2868648bf-etc-tuned\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770513    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-slash\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770545    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed9cd22c-bd90-473a-9814-eea2735c3cf0-hosts-file\") pod \"node-resolver-xhglt\" (UID: \"ed9cd22c-bd90-473a-9814-eea2735c3cf0\") " pod="openshift-dns/node-resolver-xhglt"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-system-cni-dir\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770599    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5fab6eb-e449-42a1-b535-0728a6a1be03-ovn-node-metrics-cert\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-etc-selinux\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-run-ovn\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-cnibin\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-var-lib-kubelet\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-multus-conf-dir\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.771240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.770929    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-device-dir\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.771000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znpsw\" (UniqueName: \"kubernetes.io/projected/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-kube-api-access-znpsw\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.771025    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-run-systemd\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.771098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.771125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.771190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba107ada-a3fd-4808-bfad-7105a2dc25f5-agent-certs\") pod \"konnectivity-agent-2ch2t\" (UID: \"ba107ada-a3fd-4808-bfad-7105a2dc25f5\") " pod="kube-system/konnectivity-agent-2ch2t"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.771216    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9837b724-5c2c-49df-b071-5e596f11562b-cni-binary-copy\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.771309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-node-log\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.771337    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n85x\" (UniqueName: \"kubernetes.io/projected/e5fab6eb-e449-42a1-b535-0728a6a1be03-kube-api-access-2n85x\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.771392    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-registration-dir\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.771478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd-iptables-alerter-script\") pod \"iptables-alerter-mxxg4\" (UID: \"e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd\") " pod="openshift-network-operator/iptables-alerter-mxxg4"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.771503    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd-host-slash\") pod \"iptables-alerter-mxxg4\" (UID: \"e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd\") " pod="openshift-network-operator/iptables-alerter-mxxg4"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.771954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxvnl\" (UniqueName: \"kubernetes.io/projected/e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd-kube-api-access-nxvnl\") pod \"iptables-alerter-mxxg4\" (UID: \"e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd\") " pod="openshift-network-operator/iptables-alerter-mxxg4"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.772006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vb4p\" (UniqueName: \"kubernetes.io/projected/ea3e1564-e8e9-47db-a5e3-86002afeaeab-kube-api-access-6vb4p\") pod \"node-ca-4gz27\" (UID: \"ea3e1564-e8e9-47db-a5e3-86002afeaeab\") " pod="openshift-image-registry/node-ca-4gz27"
Sep 10 20:58:28.772067 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.772048    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-cni-binary-copy\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.772802 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.772108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-multus-cni-dir\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.772802 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.772133    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-run-k8s-cni-cncf-io\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.772802 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.772148    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-systemd-units\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.772802 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.772166    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-run-netns\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.853916 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.853878    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.872828    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-etc-selinux\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.872882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-run-ovn\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.872907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-cnibin\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.872942    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-var-lib-kubelet\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.872965    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-multus-conf-dir\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-device-dir\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873016    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-run-ovn\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873029    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-cnibin\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znpsw\" (UniqueName: \"kubernetes.io/projected/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-kube-api-access-znpsw\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-var-lib-kubelet\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-multus-conf-dir\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-etc-selinux\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873116    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-device-dir\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-run-systemd\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-run-systemd\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.873203 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873208    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873239    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba107ada-a3fd-4808-bfad-7105a2dc25f5-agent-certs\") pod \"konnectivity-agent-2ch2t\" (UID: \"ba107ada-a3fd-4808-bfad-7105a2dc25f5\") " pod="kube-system/konnectivity-agent-2ch2t"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9837b724-5c2c-49df-b071-5e596f11562b-cni-binary-copy\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-node-log\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n85x\" (UniqueName: \"kubernetes.io/projected/e5fab6eb-e449-42a1-b535-0728a6a1be03-kube-api-access-2n85x\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-registration-dir\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873429    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd-iptables-alerter-script\") pod \"iptables-alerter-mxxg4\" (UID: \"e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd\") " pod="openshift-network-operator/iptables-alerter-mxxg4"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873503    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd-host-slash\") pod \"iptables-alerter-mxxg4\" (UID: \"e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd\") " pod="openshift-network-operator/iptables-alerter-mxxg4"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxvnl\" (UniqueName: \"kubernetes.io/projected/e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd-kube-api-access-nxvnl\") pod \"iptables-alerter-mxxg4\" (UID: \"e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd\") " pod="openshift-network-operator/iptables-alerter-mxxg4"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873570    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vb4p\" (UniqueName: \"kubernetes.io/projected/ea3e1564-e8e9-47db-a5e3-86002afeaeab-kube-api-access-6vb4p\") pod \"node-ca-4gz27\" (UID: \"ea3e1564-e8e9-47db-a5e3-86002afeaeab\") " pod="openshift-image-registry/node-ca-4gz27"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-cni-binary-copy\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-multus-cni-dir\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873656    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-node-log\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-run-k8s-cni-cncf-io\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.874177 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-systemd-units\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-run-netns\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873765    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-var-lib-openvswitch\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-run-openvswitch\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea3e1564-e8e9-47db-a5e3-86002afeaeab-host\") pod \"node-ca-4gz27\" (UID: \"ea3e1564-e8e9-47db-a5e3-86002afeaeab\") " pod="openshift-image-registry/node-ca-4gz27"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-registration-dir\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-system-cni-dir\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873913    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5fab6eb-e449-42a1-b535-0728a6a1be03-ovnkube-config\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0819e2f5-b35c-4836-8d5e-939990ad6009-kubelet-config\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.873977    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874001    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-systemd\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-var-lib-cni-multus\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9837b724-5c2c-49df-b071-5e596f11562b-multus-daemon-config\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72dzk\" (UniqueName: \"kubernetes.io/projected/9837b724-5c2c-49df-b071-5e596f11562b-kube-api-access-72dzk\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874159    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9837b724-5c2c-49df-b071-5e596f11562b-cni-binary-copy\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-cni-bin\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-cni-bin\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.875129 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43499b3a-6698-490b-831a-70b2868648bf-tmp\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874296    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-run-netns\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-run-multus-certs\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-socket-dir\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874416    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-multus-socket-dir-parent\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-kubelet\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c6gp\" (UniqueName: \"kubernetes.io/projected/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-kube-api-access-4c6gp\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xls6c\" (UniqueName: \"kubernetes.io/projected/ed9cd22c-bd90-473a-9814-eea2735c3cf0-kube-api-access-xls6c\") pod \"node-resolver-xhglt\" (UID: \"ed9cd22c-bd90-473a-9814-eea2735c3cf0\") " pod="openshift-dns/node-resolver-xhglt"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba107ada-a3fd-4808-bfad-7105a2dc25f5-konnectivity-ca\") pod \"konnectivity-agent-2ch2t\" (UID: \"ba107ada-a3fd-4808-bfad-7105a2dc25f5\") " pod="kube-system/konnectivity-agent-2ch2t"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-modprobe-d\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874585    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-lib-modules\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd-host-slash\") pod \"iptables-alerter-mxxg4\" (UID: \"e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd\") " pod="openshift-network-operator/iptables-alerter-mxxg4"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-hostroot\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874665    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-hostroot\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874735    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-system-cni-dir\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-etc-kubernetes\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.875933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874809    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl4gb\" (UniqueName: \"kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb\") pod \"network-check-target-pcwdg\" (UID: \"b52aa986-0758-4fd0-a230-498beba6b387\") " pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.874846    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0819e2f5-b35c-4836-8d5e-939990ad6009-dbus\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.875914    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-multus-cni-dir\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876019    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-run-netns\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ea3e1564-e8e9-47db-a5e3-86002afeaeab-serviceca\") pod \"node-ca-4gz27\" (UID: \"ea3e1564-e8e9-47db-a5e3-86002afeaeab\") " pod="openshift-image-registry/node-ca-4gz27"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zqm9\" (UniqueName: \"kubernetes.io/projected/43499b3a-6698-490b-831a-70b2868648bf-kube-api-access-5zqm9\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-etc-openvswitch\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876203    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-cni-binary-copy\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5fab6eb-e449-42a1-b535-0728a6a1be03-env-overrides\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed9cd22c-bd90-473a-9814-eea2735c3cf0-tmp-dir\") pod \"node-resolver-xhglt\" (UID: \"ed9cd22c-bd90-473a-9814-eea2735c3cf0\") " pod="openshift-dns/node-resolver-xhglt"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876243    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-run-k8s-cni-cncf-io\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876317    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-systemd-units\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd-iptables-alerter-script\") pod \"iptables-alerter-mxxg4\" (UID: \"e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd\") " pod="openshift-network-operator/iptables-alerter-mxxg4"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876414    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0819e2f5-b35c-4836-8d5e-939990ad6009-kubelet-config\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:28.876522    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:28.876642 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:28.876641    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret podName:0819e2f5-b35c-4836-8d5e-939990ad6009 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:29.376606642 +0000 UTC m=+3.056105492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret") pod "global-pull-secret-syncer-k7pdz" (UID: "0819e2f5-b35c-4836-8d5e-939990ad6009") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:28.877098 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876732    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ea3e1564-e8e9-47db-a5e3-86002afeaeab-serviceca\") pod \"node-ca-4gz27\" (UID: \"ea3e1564-e8e9-47db-a5e3-86002afeaeab\") " pod="openshift-image-registry/node-ca-4gz27"
Sep 10 20:58:28.877098 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876807    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-etc-openvswitch\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.877098 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876870    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-run-openvswitch\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.877098 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876926    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea3e1564-e8e9-47db-a5e3-86002afeaeab-host\") pod \"node-ca-4gz27\" (UID: \"ea3e1564-e8e9-47db-a5e3-86002afeaeab\") " pod="openshift-image-registry/node-ca-4gz27"
Sep 10 20:58:28.877098 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876973    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-systemd\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.877098 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.876974    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-etc-kubernetes\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.877098 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.877074    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0819e2f5-b35c-4836-8d5e-939990ad6009-dbus\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:28.877342 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.877139    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5fab6eb-e449-42a1-b535-0728a6a1be03-ovnkube-config\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.877342 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.877251    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5fab6eb-e449-42a1-b535-0728a6a1be03-env-overrides\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.877444 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.877406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-kubernetes\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.877488 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.877472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-sysctl-conf\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.877528 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.877512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-host\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.877603 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.877572    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed9cd22c-bd90-473a-9814-eea2735c3cf0-tmp-dir\") pod \"node-resolver-xhglt\" (UID: \"ed9cd22c-bd90-473a-9814-eea2735c3cf0\") " pod="openshift-dns/node-resolver-xhglt"
Sep 10 20:58:28.877685 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.877546    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-run-ovn-kubernetes\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.877800 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.877783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-var-lib-cni-multus\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.878421 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.878397    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9837b724-5c2c-49df-b071-5e596f11562b-multus-daemon-config\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.878500 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.878456    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 20:58:28.878688 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.878175    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-sysconfig\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879100    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-var-lib-openvswitch\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-sys\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-var-lib-cni-bin\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879210    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-log-socket\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5fab6eb-e449-42a1-b535-0728a6a1be03-ovnkube-script-lib\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-sys-fs\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879284    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-sysctl-conf\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-run\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879342    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-run-ovn-kubernetes\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879402    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-kubernetes\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-cnibin\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-cni-netd\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jbz4\" (UniqueName: \"kubernetes.io/projected/1bb0820e-193e-416f-a950-5eedb01fc196-kube-api-access-4jbz4\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:28.880180 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-sysctl-d\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879592    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba107ada-a3fd-4808-bfad-7105a2dc25f5-konnectivity-ca\") pod \"konnectivity-agent-2ch2t\" (UID: \"ba107ada-a3fd-4808-bfad-7105a2dc25f5\") " pod="kube-system/konnectivity-agent-2ch2t"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-var-lib-kubelet\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/43499b3a-6698-490b-831a-70b2868648bf-etc-tuned\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-host\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-slash\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed9cd22c-bd90-473a-9814-eea2735c3cf0-hosts-file\") pod \"node-resolver-xhglt\" (UID: \"ed9cd22c-bd90-473a-9814-eea2735c3cf0\") " pod="openshift-dns/node-resolver-xhglt"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879760    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-system-cni-dir\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879772    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-modprobe-d\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.879958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5fab6eb-e449-42a1-b535-0728a6a1be03-ovn-node-metrics-cert\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.880030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-sysconfig\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.880172    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-socket-dir\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.880233    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-multus-socket-dir-parent\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.880274    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-run-multus-certs\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.880891    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.881068    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-kubelet\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.882638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.881089    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-system-cni-dir\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.881024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-sys-fs\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.881037    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-slash\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.881421    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-run-netns\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.880997    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed9cd22c-bd90-473a-9814-eea2735c3cf0-hosts-file\") pod \"node-resolver-xhglt\" (UID: \"ed9cd22c-bd90-473a-9814-eea2735c3cf0\") " pod="openshift-dns/node-resolver-xhglt"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.881534    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-lib-modules\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.881188    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-run\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.880931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5fab6eb-e449-42a1-b535-0728a6a1be03-ovnkube-script-lib\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:28.881958    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:28.882045    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs podName:1bb0820e-193e-416f-a950-5eedb01fc196 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:29.382024486 +0000 UTC m=+3.061523326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs") pod "network-metrics-daemon-dkc7w" (UID: "1bb0820e-193e-416f-a950-5eedb01fc196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.882054    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.882106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-var-lib-kubelet\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.882106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-sys\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.882156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-host-cni-netd\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.882171    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-host-var-lib-cni-bin\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.882219    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/43499b3a-6698-490b-831a-70b2868648bf-etc-sysctl-d\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.882276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5fab6eb-e449-42a1-b535-0728a6a1be03-log-socket\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.882339    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9837b724-5c2c-49df-b071-5e596f11562b-cnibin\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.883485 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.883070    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba107ada-a3fd-4808-bfad-7105a2dc25f5-agent-certs\") pod \"konnectivity-agent-2ch2t\" (UID: \"ba107ada-a3fd-4808-bfad-7105a2dc25f5\") " pod="kube-system/konnectivity-agent-2ch2t"
Sep 10 20:58:28.884354 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.883602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5fab6eb-e449-42a1-b535-0728a6a1be03-ovn-node-metrics-cert\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.884477 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.884453    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/43499b3a-6698-490b-831a-70b2868648bf-tmp\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.884561 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.884542    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/43499b3a-6698-490b-831a-70b2868648bf-etc-tuned\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.885845 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.885818    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n85x\" (UniqueName: \"kubernetes.io/projected/e5fab6eb-e449-42a1-b535-0728a6a1be03-kube-api-access-2n85x\") pod \"ovnkube-node-2qkd6\" (UID: \"e5fab6eb-e449-42a1-b535-0728a6a1be03\") " pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:28.886900 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:28.886793    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:28.886900 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:28.886821    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:28.886900 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:28.886835    2580 projected.go:196] Error preparing data for projected volume kube-api-access-sl4gb for pod openshift-network-diagnostics/network-check-target-pcwdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:28.887098 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:28.886939    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb podName:b52aa986-0758-4fd0-a230-498beba6b387 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:29.386912258 +0000 UTC m=+3.066411092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sl4gb" (UniqueName: "kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb") pod "network-check-target-pcwdg" (UID: "b52aa986-0758-4fd0-a230-498beba6b387") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:28.887462 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.887443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxvnl\" (UniqueName: \"kubernetes.io/projected/e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd-kube-api-access-nxvnl\") pod \"iptables-alerter-mxxg4\" (UID: \"e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd\") " pod="openshift-network-operator/iptables-alerter-mxxg4"
Sep 10 20:58:28.888597 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.888569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znpsw\" (UniqueName: \"kubernetes.io/projected/059d85a3-7029-4c96-81b6-9a1cefe6d1ec-kube-api-access-znpsw\") pod \"multus-additional-cni-plugins-8mxtq\" (UID: \"059d85a3-7029-4c96-81b6-9a1cefe6d1ec\") " pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:28.888711 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.888632    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zqm9\" (UniqueName: \"kubernetes.io/projected/43499b3a-6698-490b-831a-70b2868648bf-kube-api-access-5zqm9\") pod \"tuned-d6fcr\" (UID: \"43499b3a-6698-490b-831a-70b2868648bf\") " pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:28.889338 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.889314    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c6gp\" (UniqueName: \"kubernetes.io/projected/9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba-kube-api-access-4c6gp\") pod \"aws-ebs-csi-driver-node-54lxl\" (UID: \"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:28.893900 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.893783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vb4p\" (UniqueName: \"kubernetes.io/projected/ea3e1564-e8e9-47db-a5e3-86002afeaeab-kube-api-access-6vb4p\") pod \"node-ca-4gz27\" (UID: \"ea3e1564-e8e9-47db-a5e3-86002afeaeab\") " pod="openshift-image-registry/node-ca-4gz27"
Sep 10 20:58:28.899329 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.899293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xls6c\" (UniqueName: \"kubernetes.io/projected/ed9cd22c-bd90-473a-9814-eea2735c3cf0-kube-api-access-xls6c\") pod \"node-resolver-xhglt\" (UID: \"ed9cd22c-bd90-473a-9814-eea2735c3cf0\") " pod="openshift-dns/node-resolver-xhglt"
Sep 10 20:58:28.900964 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.900929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72dzk\" (UniqueName: \"kubernetes.io/projected/9837b724-5c2c-49df-b071-5e596f11562b-kube-api-access-72dzk\") pod \"multus-h9lm2\" (UID: \"9837b724-5c2c-49df-b071-5e596f11562b\") " pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:28.901094 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.900937    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jbz4\" (UniqueName: \"kubernetes.io/projected/1bb0820e-193e-416f-a950-5eedb01fc196-kube-api-access-4jbz4\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:28.910602 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.910578    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cljzt"
Sep 10 20:58:28.920646 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.920612    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cljzt"
Sep 10 20:58:28.954955 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.954896    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal" event={"ID":"ed1b1f179b2e10e70b3c97f4374d819e","Type":"ContainerStarted","Data":"fe25ca304c33f665d80bbb3d5579179ea9cda6a856fd90400d3dc2eda917fccd"}
Sep 10 20:58:28.956351 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:28.956325    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-142.ec2.internal" event={"ID":"281550d9d955d980d569b392e95bf98a","Type":"ContainerStarted","Data":"58cb81deb8dcd21259af3e0ee34050514fef47f9cceb42d647e890afef37e1ba"}
Sep 10 20:58:29.036750 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.036651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4gz27"
Sep 10 20:58:29.044144 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.044120    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:29.054271 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.054245    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2ch2t"
Sep 10 20:58:29.063336 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.063316    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl"
Sep 10 20:58:29.072361 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.072334    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d6fcr"
Sep 10 20:58:29.079283 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.079262    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xhglt"
Sep 10 20:58:29.088353 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.088330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8mxtq"
Sep 10 20:58:29.098289 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.098269    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mxxg4"
Sep 10 20:58:29.107240 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.107214    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:29.112685 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.112663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h9lm2"
Sep 10 20:58:29.384554 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.384465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:29.384554 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.384530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:29.384796 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:29.384649    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:29.384796 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:29.384651    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:29.384796 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:29.384708    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret podName:0819e2f5-b35c-4836-8d5e-939990ad6009 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:30.384693945 +0000 UTC m=+4.064192776 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret") pod "global-pull-secret-syncer-k7pdz" (UID: "0819e2f5-b35c-4836-8d5e-939990ad6009") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:29.384796 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:29.384722    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs podName:1bb0820e-193e-416f-a950-5eedb01fc196 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:30.384715495 +0000 UTC m=+4.064214325 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs") pod "network-metrics-daemon-dkc7w" (UID: "1bb0820e-193e-416f-a950-5eedb01fc196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:29.485450 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.485413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl4gb\" (UniqueName: \"kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb\") pod \"network-check-target-pcwdg\" (UID: \"b52aa986-0758-4fd0-a230-498beba6b387\") " pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:29.485647 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:29.485601    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:29.485647 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:29.485629    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:29.485647 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:29.485642    2580 projected.go:196] Error preparing data for projected volume kube-api-access-sl4gb for pod openshift-network-diagnostics/network-check-target-pcwdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:29.485792 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:29.485715    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb podName:b52aa986-0758-4fd0-a230-498beba6b387 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:30.48569295 +0000 UTC m=+4.165191793 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sl4gb" (UniqueName: "kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb") pod "network-check-target-pcwdg" (UID: "b52aa986-0758-4fd0-a230-498beba6b387") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:29.924764 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.924724    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 20:53:29 +0000 UTC" deadline="2028-05-27 14:47:36.157163166 +0000 UTC"
Sep 10 20:58:29.924764 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.924757    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14993h49m6.232409816s"
Sep 10 20:58:29.950402 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.950317    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:29.950806 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:29.950317    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:29.950806 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:29.950556    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:29.950806 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:29.950646    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:30.017084 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:30.017051    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43499b3a_6698_490b_831a_70b2868648bf.slice/crio-e7baba5ea8e39fa475c09397b808caf8b2ec84b6f64933176daafda44e9d7982 WatchSource:0}: Error finding container e7baba5ea8e39fa475c09397b808caf8b2ec84b6f64933176daafda44e9d7982: Status 404 returned error can't find the container with id e7baba5ea8e39fa475c09397b808caf8b2ec84b6f64933176daafda44e9d7982
Sep 10 20:58:30.017847 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:30.017820    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4be32c3_bb8a_4bb3_a541_1b836e5f9dfd.slice/crio-a5b2f8b04d22929b6d49b8e3a9f2fc0df4082074ec8d62b455fef553ac2d67af WatchSource:0}: Error finding container a5b2f8b04d22929b6d49b8e3a9f2fc0df4082074ec8d62b455fef553ac2d67af: Status 404 returned error can't find the container with id a5b2f8b04d22929b6d49b8e3a9f2fc0df4082074ec8d62b455fef553ac2d67af
Sep 10 20:58:30.022203 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:30.022178    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod059d85a3_7029_4c96_81b6_9a1cefe6d1ec.slice/crio-ce79008f75f28e0dd084a3f95c3a4dc3939403e213484a84e911a4e11800c57f WatchSource:0}: Error finding container ce79008f75f28e0dd084a3f95c3a4dc3939403e213484a84e911a4e11800c57f: Status 404 returned error can't find the container with id ce79008f75f28e0dd084a3f95c3a4dc3939403e213484a84e911a4e11800c57f
Sep 10 20:58:30.024686 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:30.024648    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5fab6eb_e449_42a1_b535_0728a6a1be03.slice/crio-5c458f25bf39ed0de7c27185ec483fa7ce910196be5236cf328abe747d6daf17 WatchSource:0}: Error finding container 5c458f25bf39ed0de7c27185ec483fa7ce910196be5236cf328abe747d6daf17: Status 404 returned error can't find the container with id 5c458f25bf39ed0de7c27185ec483fa7ce910196be5236cf328abe747d6daf17
Sep 10 20:58:30.025905 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:30.025875    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea3e1564_e8e9_47db_a5e3_86002afeaeab.slice/crio-d757e5049a8c21966346c3444a90d8b74733802dd6a4d1207c3e55b5b1a82a16 WatchSource:0}: Error finding container d757e5049a8c21966346c3444a90d8b74733802dd6a4d1207c3e55b5b1a82a16: Status 404 returned error can't find the container with id d757e5049a8c21966346c3444a90d8b74733802dd6a4d1207c3e55b5b1a82a16
Sep 10 20:58:30.027400 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:30.027361    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9837b724_5c2c_49df_b071_5e596f11562b.slice/crio-75e157a1ee2c73034f2649ad1a26755a94f136b12838dfcd21a9c2e75bcd380b WatchSource:0}: Error finding container 75e157a1ee2c73034f2649ad1a26755a94f136b12838dfcd21a9c2e75bcd380b: Status 404 returned error can't find the container with id 75e157a1ee2c73034f2649ad1a26755a94f136b12838dfcd21a9c2e75bcd380b
Sep 10 20:58:30.029043 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:30.028782    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded9cd22c_bd90_473a_9814_eea2735c3cf0.slice/crio-f5830213f98d8a994d96ad707a21d05ab1984a053c04324fa034b920ae2e096e WatchSource:0}: Error finding container f5830213f98d8a994d96ad707a21d05ab1984a053c04324fa034b920ae2e096e: Status 404 returned error can't find the container with id f5830213f98d8a994d96ad707a21d05ab1984a053c04324fa034b920ae2e096e
Sep 10 20:58:30.030263 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:30.030242    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb3e9fd_e0aa_44a8_8b88_a4cf5df5cfba.slice/crio-eb410dd928e0275de6aa5d8be40ed465f2dd653535c0747939e95b2138bee144 WatchSource:0}: Error finding container eb410dd928e0275de6aa5d8be40ed465f2dd653535c0747939e95b2138bee144: Status 404 returned error can't find the container with id eb410dd928e0275de6aa5d8be40ed465f2dd653535c0747939e95b2138bee144
Sep 10 20:58:30.034489 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:58:30.032517    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba107ada_a3fd_4808_bfad_7105a2dc25f5.slice/crio-305f3cf714d5447b01aa46a18e581cb851ca611cd27644dab946a6c7f7d5e06d WatchSource:0}: Error finding container 305f3cf714d5447b01aa46a18e581cb851ca611cd27644dab946a6c7f7d5e06d: Status 404 returned error can't find the container with id 305f3cf714d5447b01aa46a18e581cb851ca611cd27644dab946a6c7f7d5e06d
Sep 10 20:58:30.394306 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:30.393911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:30.394306 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:30.393992    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:30.394306 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:30.394046    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:30.394306 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:30.394118    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:30.394306 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:30.394129    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs podName:1bb0820e-193e-416f-a950-5eedb01fc196 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:32.394108349 +0000 UTC m=+6.073607190 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs") pod "network-metrics-daemon-dkc7w" (UID: "1bb0820e-193e-416f-a950-5eedb01fc196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:30.394306 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:30.394158    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret podName:0819e2f5-b35c-4836-8d5e-939990ad6009 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:32.394148304 +0000 UTC m=+6.073647137 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret") pod "global-pull-secret-syncer-k7pdz" (UID: "0819e2f5-b35c-4836-8d5e-939990ad6009") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:30.494931 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:30.494826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl4gb\" (UniqueName: \"kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb\") pod \"network-check-target-pcwdg\" (UID: \"b52aa986-0758-4fd0-a230-498beba6b387\") " pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:30.495111 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:30.495005    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:30.495111 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:30.495027    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:30.495111 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:30.495039    2580 projected.go:196] Error preparing data for projected volume kube-api-access-sl4gb for pod openshift-network-diagnostics/network-check-target-pcwdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:30.495273 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:30.495122    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb podName:b52aa986-0758-4fd0-a230-498beba6b387 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:32.495103489 +0000 UTC m=+6.174602340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sl4gb" (UniqueName: "kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb") pod "network-check-target-pcwdg" (UID: "b52aa986-0758-4fd0-a230-498beba6b387") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:30.925031 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:30.924873    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 20:53:29 +0000 UTC" deadline="2028-04-09 06:22:24.51276323 +0000 UTC"
Sep 10 20:58:30.925031 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:30.924912    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13833h23m53.587855236s"
Sep 10 20:58:30.951199 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:30.951172    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:30.951345 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:30.951307    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:30.972939 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:30.972909    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl" event={"ID":"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba","Type":"ContainerStarted","Data":"eb410dd928e0275de6aa5d8be40ed465f2dd653535c0747939e95b2138bee144"}
Sep 10 20:58:30.975011 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:30.974977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h9lm2" event={"ID":"9837b724-5c2c-49df-b071-5e596f11562b","Type":"ContainerStarted","Data":"75e157a1ee2c73034f2649ad1a26755a94f136b12838dfcd21a9c2e75bcd380b"}
Sep 10 20:58:30.982838 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:30.982757    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4gz27" event={"ID":"ea3e1564-e8e9-47db-a5e3-86002afeaeab","Type":"ContainerStarted","Data":"d757e5049a8c21966346c3444a90d8b74733802dd6a4d1207c3e55b5b1a82a16"}
Sep 10 20:58:30.996435 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:30.996296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6" event={"ID":"e5fab6eb-e449-42a1-b535-0728a6a1be03","Type":"ContainerStarted","Data":"5c458f25bf39ed0de7c27185ec483fa7ce910196be5236cf328abe747d6daf17"}
Sep 10 20:58:31.006830 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:31.006765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mxtq" event={"ID":"059d85a3-7029-4c96-81b6-9a1cefe6d1ec","Type":"ContainerStarted","Data":"ce79008f75f28e0dd084a3f95c3a4dc3939403e213484a84e911a4e11800c57f"}
Sep 10 20:58:31.017548 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:31.017486    2580 generic.go:356] "Generic (PLEG): container finished" podID="ed1b1f179b2e10e70b3c97f4374d819e" containerID="26148dab1ca1c7cd5d0a1f74f160dac7219e9e1c7c1e6f894b3222bea6c39ff6" exitCode=0
Sep 10 20:58:31.017677 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:31.017580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal" event={"ID":"ed1b1f179b2e10e70b3c97f4374d819e","Type":"ContainerDied","Data":"26148dab1ca1c7cd5d0a1f74f160dac7219e9e1c7c1e6f894b3222bea6c39ff6"}
Sep 10 20:58:31.035169 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:31.035033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mxxg4" event={"ID":"e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd","Type":"ContainerStarted","Data":"a5b2f8b04d22929b6d49b8e3a9f2fc0df4082074ec8d62b455fef553ac2d67af"}
Sep 10 20:58:31.048953 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:31.048919    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d6fcr" event={"ID":"43499b3a-6698-490b-831a-70b2868648bf","Type":"ContainerStarted","Data":"e7baba5ea8e39fa475c09397b808caf8b2ec84b6f64933176daafda44e9d7982"}
Sep 10 20:58:31.059809 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:31.059776    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-142.ec2.internal" event={"ID":"281550d9d955d980d569b392e95bf98a","Type":"ContainerStarted","Data":"e3801eecbf786e85babd3f981be18f810f460dc843b77f019211a0d898e8ced3"}
Sep 10 20:58:31.070590 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:31.070558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2ch2t" event={"ID":"ba107ada-a3fd-4808-bfad-7105a2dc25f5","Type":"ContainerStarted","Data":"305f3cf714d5447b01aa46a18e581cb851ca611cd27644dab946a6c7f7d5e06d"}
Sep 10 20:58:31.077713 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:31.077661    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-142.ec2.internal" podStartSLOduration=3.077641344 podStartE2EDuration="3.077641344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:31.07698524 +0000 UTC m=+4.756484093" watchObservedRunningTime="2026-09-10 20:58:31.077641344 +0000 UTC m=+4.757140195"
Sep 10 20:58:31.080297 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:31.080233    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xhglt" event={"ID":"ed9cd22c-bd90-473a-9814-eea2735c3cf0","Type":"ContainerStarted","Data":"f5830213f98d8a994d96ad707a21d05ab1984a053c04324fa034b920ae2e096e"}
Sep 10 20:58:31.950291 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:31.950209    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:31.950841 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:31.950340    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:31.950841 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:31.950733    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:31.950951 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:31.950848    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:32.105369 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:32.105334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal" event={"ID":"ed1b1f179b2e10e70b3c97f4374d819e","Type":"ContainerStarted","Data":"8e93a7de432f7b1ab644fac43562091bb11190f8069d59ebb0c03b9e6f8e35d9"}
Sep 10 20:58:32.412462 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:32.412085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:32.412462 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:32.412170    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:32.412462 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:32.412217    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:32.412462 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:32.412296    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret podName:0819e2f5-b35c-4836-8d5e-939990ad6009 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:36.41227531 +0000 UTC m=+10.091774165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret") pod "global-pull-secret-syncer-k7pdz" (UID: "0819e2f5-b35c-4836-8d5e-939990ad6009") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:32.412462 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:32.412302    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:32.412462 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:32.412371    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs podName:1bb0820e-193e-416f-a950-5eedb01fc196 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:36.41235511 +0000 UTC m=+10.091853943 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs") pod "network-metrics-daemon-dkc7w" (UID: "1bb0820e-193e-416f-a950-5eedb01fc196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:32.513376 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:32.513338    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl4gb\" (UniqueName: \"kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb\") pod \"network-check-target-pcwdg\" (UID: \"b52aa986-0758-4fd0-a230-498beba6b387\") " pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:32.513563 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:32.513538    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:32.513563 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:32.513562    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:32.513695 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:32.513575    2580 projected.go:196] Error preparing data for projected volume kube-api-access-sl4gb for pod openshift-network-diagnostics/network-check-target-pcwdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:32.513695 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:32.513632    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb podName:b52aa986-0758-4fd0-a230-498beba6b387 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:36.513613573 +0000 UTC m=+10.193112407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sl4gb" (UniqueName: "kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb") pod "network-check-target-pcwdg" (UID: "b52aa986-0758-4fd0-a230-498beba6b387") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:32.952516 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:32.952479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:32.952980 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:32.952613    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:33.950421 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:33.950381    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:33.950616 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:33.950520    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:33.951119 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:33.950943    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:33.951119 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:33.951051    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:34.950887 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:34.950848    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:34.951363 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:34.950995    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:35.950600 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:35.950384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:35.950796 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:35.950452    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:35.950796 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:35.950704    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:35.950916 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:35.950793    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:36.449983 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:36.449896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:36.450162 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:36.449983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:36.450162 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:36.450095    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:36.450162 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:36.450153    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs podName:1bb0820e-193e-416f-a950-5eedb01fc196 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:44.450135486 +0000 UTC m=+18.129634334 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs") pod "network-metrics-daemon-dkc7w" (UID: "1bb0820e-193e-416f-a950-5eedb01fc196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:36.450635 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:36.450095    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:36.450763 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:36.450661    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret podName:0819e2f5-b35c-4836-8d5e-939990ad6009 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:44.450644771 +0000 UTC m=+18.130143603 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret") pod "global-pull-secret-syncer-k7pdz" (UID: "0819e2f5-b35c-4836-8d5e-939990ad6009") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:36.551013 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:36.550488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl4gb\" (UniqueName: \"kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb\") pod \"network-check-target-pcwdg\" (UID: \"b52aa986-0758-4fd0-a230-498beba6b387\") " pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:36.551013 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:36.550670    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:36.551013 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:36.550693    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:36.551013 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:36.550708    2580 projected.go:196] Error preparing data for projected volume kube-api-access-sl4gb for pod openshift-network-diagnostics/network-check-target-pcwdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:36.551013 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:36.550782    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb podName:b52aa986-0758-4fd0-a230-498beba6b387 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:44.550759537 +0000 UTC m=+18.230258365 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sl4gb" (UniqueName: "kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb") pod "network-check-target-pcwdg" (UID: "b52aa986-0758-4fd0-a230-498beba6b387") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:36.952417 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:36.951939    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:36.952417 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:36.952086    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:37.950237 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:37.950199    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:37.950412 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:37.950337    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:37.950791 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:37.950767    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:37.950936 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:37.950879    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:38.950628 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:38.950595    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:38.951074 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:38.950734    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:39.950390 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:39.950349    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:39.950553 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:39.950486    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:39.950553 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:39.950539    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:39.950685 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:39.950645    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:40.949842 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:40.949805    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:40.950004 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:40.949948    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:41.949862 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:41.949831    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:41.950281 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:41.949873    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:41.950281 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:41.949944    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:41.950281 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:41.950084    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:42.950196 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:42.950157    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:42.950676 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:42.950289    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:43.950111 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:43.950070    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:43.950111 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:43.950092    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:43.950602 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:43.950224    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:43.950602 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:43.950259    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:44.506400 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:44.506314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:44.506400 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:44.506373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:44.506617 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:44.506489    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:44.506617 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:44.506552    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret podName:0819e2f5-b35c-4836-8d5e-939990ad6009 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:00.506535165 +0000 UTC m=+34.186033996 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret") pod "global-pull-secret-syncer-k7pdz" (UID: "0819e2f5-b35c-4836-8d5e-939990ad6009") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:44.506617 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:44.506487    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:44.506768 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:44.506642    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs podName:1bb0820e-193e-416f-a950-5eedb01fc196 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:00.506625904 +0000 UTC m=+34.186124731 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs") pod "network-metrics-daemon-dkc7w" (UID: "1bb0820e-193e-416f-a950-5eedb01fc196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:44.607694 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:44.607664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl4gb\" (UniqueName: \"kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb\") pod \"network-check-target-pcwdg\" (UID: \"b52aa986-0758-4fd0-a230-498beba6b387\") " pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:44.607865 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:44.607818    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:44.607865 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:44.607839    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:44.607865 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:44.607850    2580 projected.go:196] Error preparing data for projected volume kube-api-access-sl4gb for pod openshift-network-diagnostics/network-check-target-pcwdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:44.608034 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:44.607911    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb podName:b52aa986-0758-4fd0-a230-498beba6b387 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:00.607893064 +0000 UTC m=+34.287391897 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sl4gb" (UniqueName: "kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb") pod "network-check-target-pcwdg" (UID: "b52aa986-0758-4fd0-a230-498beba6b387") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:44.950689 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:44.950612    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:44.951096 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:44.950757    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:45.950019 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:45.949986    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:45.950223 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:45.949993    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:45.950223 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:45.950119    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:45.950223 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:45.950204    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:46.950870 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:46.950835    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:46.951306 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:46.950932    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:47.950719 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:47.950686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:47.950850 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:47.950686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:47.950900 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:47.950774    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:47.950900 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:47.950864    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:48.950196 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:48.949897    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:48.950371 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:48.950263    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:49.140728 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.140689    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2ch2t" event={"ID":"ba107ada-a3fd-4808-bfad-7105a2dc25f5","Type":"ContainerStarted","Data":"21cb86604ff7e9bd571b5f68d709181fa2c3c4aa141c6535b3503f0825a6a626"}
Sep 10 20:58:49.142214 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.142191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xhglt" event={"ID":"ed9cd22c-bd90-473a-9814-eea2735c3cf0","Type":"ContainerStarted","Data":"0bf3a25772b289e1bfeb94f93d8d7974af06926a0ca90df66d1bc281adff941e"}
Sep 10 20:58:49.143736 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.143711    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl" event={"ID":"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba","Type":"ContainerStarted","Data":"8549228badb3f51373e8b6577181f22d91193a7983cdb0834e420dfd85d0d5ef"}
Sep 10 20:58:49.145455 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.145435    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h9lm2" event={"ID":"9837b724-5c2c-49df-b071-5e596f11562b","Type":"ContainerStarted","Data":"838001ec428adc345e7e1048ed89724e83d8a901347ef365ac74455cab9c124f"}
Sep 10 20:58:49.146787 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.146767    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4gz27" event={"ID":"ea3e1564-e8e9-47db-a5e3-86002afeaeab","Type":"ContainerStarted","Data":"c6760753ea98cd3126d85ed7de58861fb05127db23f0e70defa781c2f6112458"}
Sep 10 20:58:49.149539 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.149521    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6" event={"ID":"e5fab6eb-e449-42a1-b535-0728a6a1be03","Type":"ContainerStarted","Data":"c0a339458f64c7edd1ac9ada400309c28792acf4b99c745ee618aafd85700ad2"}
Sep 10 20:58:49.149629 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.149578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6" event={"ID":"e5fab6eb-e449-42a1-b535-0728a6a1be03","Type":"ContainerStarted","Data":"20e7f931b55e04c338078f169f6687d33e2caf5e7ef6dec60daf25801fdce360"}
Sep 10 20:58:49.149629 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.149618    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6" event={"ID":"e5fab6eb-e449-42a1-b535-0728a6a1be03","Type":"ContainerStarted","Data":"c7e93c25fe4c6be50314bdf70427ed176f768374adb5c63618884541a3bd4a88"}
Sep 10 20:58:49.149711 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.149657    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6" event={"ID":"e5fab6eb-e449-42a1-b535-0728a6a1be03","Type":"ContainerStarted","Data":"b10afac74be9e84b21b4ee5e84336a93e5c4419ffa4931cd3b6e059b9e2d5d49"}
Sep 10 20:58:49.149711 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.149684    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6" event={"ID":"e5fab6eb-e449-42a1-b535-0728a6a1be03","Type":"ContainerStarted","Data":"af3d2c1e2f91f8d4c5a02b578c464de227dc7667b54c6c0c1ba36063aa796c1b"}
Sep 10 20:58:49.149788 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.149711    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6" event={"ID":"e5fab6eb-e449-42a1-b535-0728a6a1be03","Type":"ContainerStarted","Data":"2c4c0b3793ea3117f111e73043ffbbd353050e34b9553213204e7ed75012987c"}
Sep 10 20:58:49.150941 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.150922    2580 generic.go:356] "Generic (PLEG): container finished" podID="059d85a3-7029-4c96-81b6-9a1cefe6d1ec" containerID="e4ccaac469b21e4fafbecb7658c065675f5e04090a739296358bf1216e4b92a0" exitCode=0
Sep 10 20:58:49.151015 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.150988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mxtq" event={"ID":"059d85a3-7029-4c96-81b6-9a1cefe6d1ec","Type":"ContainerDied","Data":"e4ccaac469b21e4fafbecb7658c065675f5e04090a739296358bf1216e4b92a0"}
Sep 10 20:58:49.153801 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.152655    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d6fcr" event={"ID":"43499b3a-6698-490b-831a-70b2868648bf","Type":"ContainerStarted","Data":"1399499acedb8f694cbcdb29ce9a027f3dc2937882064b39e12a10fdb1f69689"}
Sep 10 20:58:49.156962 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.156926    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-142.ec2.internal" podStartSLOduration=21.156912165 podStartE2EDuration="21.156912165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:32.125658188 +0000 UTC m=+5.805157039" watchObservedRunningTime="2026-09-10 20:58:49.156912165 +0000 UTC m=+22.836411014"
Sep 10 20:58:49.157161 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.157137    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2ch2t" podStartSLOduration=7.980416286 podStartE2EDuration="22.157131066s" totalImagesPullingTime="14.17671478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:49.156973262 +0000 UTC m=+22.836472112" watchObservedRunningTime="2026-09-10 20:58:49.157131066 +0000 UTC m=+22.836629915"
Sep 10 20:58:49.210791 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.210749    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h9lm2" podStartSLOduration=4.149935642 podStartE2EDuration="22.210735868s" totalImagesPullingTime="18.060800226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:49.210502152 +0000 UTC m=+22.890001001" watchObservedRunningTime="2026-09-10 20:58:49.210735868 +0000 UTC m=+22.890234719"
Sep 10 20:58:49.211040 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.211021    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xhglt" podStartSLOduration=4.22584136 podStartE2EDuration="22.211014868s" totalImagesPullingTime="17.985173508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:49.192962152 +0000 UTC m=+22.872461001" watchObservedRunningTime="2026-09-10 20:58:49.211014868 +0000 UTC m=+22.890513717"
Sep 10 20:58:49.224757 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.224726    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4gz27" podStartSLOduration=4.236104715 podStartE2EDuration="22.224714005s" totalImagesPullingTime="17.98860929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:49.224496821 +0000 UTC m=+22.903995670" watchObservedRunningTime="2026-09-10 20:58:49.224714005 +0000 UTC m=+22.904212854"
Sep 10 20:58:49.242708 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.242673    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-d6fcr" podStartSLOduration=4.228883838 podStartE2EDuration="22.242663037s" totalImagesPullingTime="18.013779199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:49.242597565 +0000 UTC m=+22.922096414" watchObservedRunningTime="2026-09-10 20:58:49.242663037 +0000 UTC m=+22.922161887"
Sep 10 20:58:49.484629 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.484483    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 20:58:49.930553 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.930361    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T20:58:49.484624704Z","UUID":"d3847f26-23a2-4836-967d-9b8d1bc26f21","Handler":null,"Name":"","Endpoint":""}
Sep 10 20:58:49.932294 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.932268    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 20:58:49.932429 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.932305    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 20:58:49.950572 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.950274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:49.950572 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:49.950390    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:49.950572 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:49.950546    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:49.950825 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:49.950641    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:50.157765 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:50.157726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl" event={"ID":"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba","Type":"ContainerStarted","Data":"d58032e9c093a3312d5961de7b1f8507eea1937db1202d32228220b0cd3d8052"}
Sep 10 20:58:50.159452 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:50.159424    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mxxg4" event={"ID":"e4be32c3-bb8a-4bb3-a541-1b836e5f9dfd","Type":"ContainerStarted","Data":"644b218d3f7945cce463ec74a48590b8fd7ab64b1770ef6e555b14b3311bbba9"}
Sep 10 20:58:50.177413 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:50.177366    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mxxg4" podStartSLOduration=5.174582831 podStartE2EDuration="23.177348794s" totalImagesPullingTime="18.002765963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:50.176981355 +0000 UTC m=+23.856480208" watchObservedRunningTime="2026-09-10 20:58:50.177348794 +0000 UTC m=+23.856847644"
Sep 10 20:58:50.950236 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:50.950149    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:50.950393 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:50.950294    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:51.166304 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:51.166267    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6" event={"ID":"e5fab6eb-e449-42a1-b535-0728a6a1be03","Type":"ContainerStarted","Data":"2e27a98e7c7d538b4d3fbb48fc71395f90694c963d0926304dec69698095e1e5"}
Sep 10 20:58:51.168720 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:51.168691    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl" event={"ID":"9bb3e9fd-e0aa-44a8-8b88-a4cf5df5cfba","Type":"ContainerStarted","Data":"7dd0f66dc04506499cefb8054709f46c257b09112f1c64dadd3831aaa4cd629f"}
Sep 10 20:58:51.950652 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:51.950617    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:51.950652 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:51.950647    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:51.950883 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:51.950744    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:51.950883 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:51.950860    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:52.679552 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:52.679521    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2ch2t"
Sep 10 20:58:52.680185 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:52.680167    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2ch2t"
Sep 10 20:58:52.697195 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:52.697154    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-54lxl" podStartSLOduration=5.503669749 podStartE2EDuration="25.697140085s" totalImagesPullingTime="20.193470336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:51.188476026 +0000 UTC m=+24.867974876" watchObservedRunningTime="2026-09-10 20:58:52.697140085 +0000 UTC m=+26.376638934"
Sep 10 20:58:52.950164 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:52.950097    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:52.950316 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:52.950205    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:53.181849 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:53.181452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6" event={"ID":"e5fab6eb-e449-42a1-b535-0728a6a1be03","Type":"ContainerStarted","Data":"e98aa0d9e60e13cb376973447e5e23819ee14e37adba2131e5b6706bf8a6d5a5"}
Sep 10 20:58:53.181849 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:53.181544    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2ch2t"
Sep 10 20:58:53.181849 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:53.181775    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:53.182772 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:53.182011    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:53.182772 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:53.182117    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2ch2t"
Sep 10 20:58:53.200978 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:53.200934    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:53.201052 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:53.201027    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:53.213206 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:53.213171    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6" podStartSLOduration=8.163396895 podStartE2EDuration="26.213157973s" totalImagesPullingTime="18.049761078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:53.212861458 +0000 UTC m=+26.892360308" watchObservedRunningTime="2026-09-10 20:58:53.213157973 +0000 UTC m=+26.892656823"
Sep 10 20:58:53.283000 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:53.282874    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:58:53.950719 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:53.950681    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:53.950719 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:53.950718    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:53.951323 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:53.950812    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:53.951323 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:53.950923    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:54.185194 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:54.185155    2580 generic.go:356] "Generic (PLEG): container finished" podID="059d85a3-7029-4c96-81b6-9a1cefe6d1ec" containerID="63b9d9bf3fe886bd082e3174aa52625d6997440478bd40369bff18764af6c1b2" exitCode=0
Sep 10 20:58:54.185359 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:54.185247    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mxtq" event={"ID":"059d85a3-7029-4c96-81b6-9a1cefe6d1ec","Type":"ContainerDied","Data":"63b9d9bf3fe886bd082e3174aa52625d6997440478bd40369bff18764af6c1b2"}
Sep 10 20:58:54.949856 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:54.949828    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:54.949979 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:54.949945    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:55.069957 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:55.069871    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pcwdg"]
Sep 10 20:58:55.070623 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:55.069988    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:55.070623 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:55.070094    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:55.073045 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:55.073020    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k7pdz"]
Sep 10 20:58:55.073686 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:55.073662    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dkc7w"]
Sep 10 20:58:55.073783 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:55.073744    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:55.073843 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:55.073823    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:55.190122 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:55.190086    2580 generic.go:356] "Generic (PLEG): container finished" podID="059d85a3-7029-4c96-81b6-9a1cefe6d1ec" containerID="f8687034bcb8558bf7f153e774e10d79483e3bcdd4963fc7a3fd0938c3397c6a" exitCode=0
Sep 10 20:58:55.190251 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:55.190124    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mxtq" event={"ID":"059d85a3-7029-4c96-81b6-9a1cefe6d1ec","Type":"ContainerDied","Data":"f8687034bcb8558bf7f153e774e10d79483e3bcdd4963fc7a3fd0938c3397c6a"}
Sep 10 20:58:55.190310 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:55.190288    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:55.190426 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:55.190406    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:56.197174 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:56.197096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mxtq" event={"ID":"059d85a3-7029-4c96-81b6-9a1cefe6d1ec","Type":"ContainerDied","Data":"ca1cf5124254ef74b74b12f827e599d79642e18eac0701256e5555059bbfd196"}
Sep 10 20:58:56.197652 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:56.197079    2580 generic.go:356] "Generic (PLEG): container finished" podID="059d85a3-7029-4c96-81b6-9a1cefe6d1ec" containerID="ca1cf5124254ef74b74b12f827e599d79642e18eac0701256e5555059bbfd196" exitCode=0
Sep 10 20:58:56.951131 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:56.950817    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:56.951131 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:56.950899    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:56.951131 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:56.950939    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:56.951131 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:56.950934    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:56.951131 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:56.951000    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:58:56.951131 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:56.951099    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:58.950316 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:58.950093    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:58:58.950857 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:58.950096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:58:58.950857 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:58:58.950149    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:58:58.950857 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:58.950431    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkc7w" podUID="1bb0820e-193e-416f-a950-5eedb01fc196"
Sep 10 20:58:58.950857 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:58.950497    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pcwdg" podUID="b52aa986-0758-4fd0-a230-498beba6b387"
Sep 10 20:58:58.950857 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:58:58.950565    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k7pdz" podUID="0819e2f5-b35c-4836-8d5e-939990ad6009"
Sep 10 20:59:00.525204 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.525172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:59:00.525531 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.525260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:59:00.525531 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:00.525307    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:59:00.525531 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:00.525366    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs podName:1bb0820e-193e-416f-a950-5eedb01fc196 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:32.525350246 +0000 UTC m=+66.204849092 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs") pod "network-metrics-daemon-dkc7w" (UID: "1bb0820e-193e-416f-a950-5eedb01fc196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:59:00.525531 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:00.525380    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:59:00.525531 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:00.525435    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret podName:0819e2f5-b35c-4836-8d5e-939990ad6009 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:32.525416576 +0000 UTC m=+66.204915427 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret") pod "global-pull-secret-syncer-k7pdz" (UID: "0819e2f5-b35c-4836-8d5e-939990ad6009") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:59:00.625733 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.625702    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl4gb\" (UniqueName: \"kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb\") pod \"network-check-target-pcwdg\" (UID: \"b52aa986-0758-4fd0-a230-498beba6b387\") " pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:59:00.625936 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:00.625913    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:59:00.625980 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:00.625945    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:59:00.625980 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:00.625960    2580 projected.go:196] Error preparing data for projected volume kube-api-access-sl4gb for pod openshift-network-diagnostics/network-check-target-pcwdg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:59:00.626050 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:00.626030    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb podName:b52aa986-0758-4fd0-a230-498beba6b387 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:32.626010053 +0000 UTC m=+66.305508900 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sl4gb" (UniqueName: "kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb") pod "network-check-target-pcwdg" (UID: "b52aa986-0758-4fd0-a230-498beba6b387") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:59:00.640773 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.640755    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-142.ec2.internal" event="NodeReady"
Sep 10 20:59:00.640890 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.640880    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 20:59:00.677134 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.677076    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75555b558c-68ncn"]
Sep 10 20:59:00.683745 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.683727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.686856 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.686661    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 20:59:00.686856 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.686735    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 20:59:00.686856 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.686751    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-k4c27\""
Sep 10 20:59:00.687016 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.686966    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 20:59:00.691779 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.691761    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 20:59:00.701742 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.701719    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h2hwr"]
Sep 10 20:59:00.714347 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.714316    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75555b558c-68ncn"]
Sep 10 20:59:00.714480 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.714352    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6ntcm"]
Sep 10 20:59:00.714480 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.714428    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:00.717498 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.717243    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 20:59:00.717498 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.717282    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 20:59:00.717662 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.717580    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7j9n7\""
Sep 10 20:59:00.722915 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.722883    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6ntcm"]
Sep 10 20:59:00.723019 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.722930    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h2hwr"]
Sep 10 20:59:00.723105 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.723030    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6ntcm"
Sep 10 20:59:00.725837 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.725818    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 20:59:00.725946 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.725812    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 20:59:00.726010 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.725995    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9vpqg\""
Sep 10 20:59:00.726103 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.726090    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 20:59:00.827309 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.827554 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9678ae86-ae81-4ade-8164-8f491017c3d2-ca-trust-extracted\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.827554 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-certificates\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.827554 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9678ae86-ae81-4ade-8164-8f491017c3d2-image-registry-private-configuration\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.827554 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9678ae86-ae81-4ade-8164-8f491017c3d2-trusted-ca\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.827736 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827557    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-bound-sa-token\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.827736 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827590    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c7d151d7-5a60-43b0-bb19-b89cdb84f169-config-volume\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:00.827736 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827613    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:00.827736 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827657    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8df9efce-a745-452e-a40d-60a5463e4537-cert\") pod \"ingress-canary-6ntcm\" (UID: \"8df9efce-a745-452e-a40d-60a5463e4537\") " pod="openshift-ingress-canary/ingress-canary-6ntcm"
Sep 10 20:59:00.827736 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcwph\" (UniqueName: \"kubernetes.io/projected/8df9efce-a745-452e-a40d-60a5463e4537-kube-api-access-lcwph\") pod \"ingress-canary-6ntcm\" (UID: \"8df9efce-a745-452e-a40d-60a5463e4537\") " pod="openshift-ingress-canary/ingress-canary-6ntcm"
Sep 10 20:59:00.827736 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9678ae86-ae81-4ade-8164-8f491017c3d2-installation-pull-secrets\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.827976 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjvjt\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-kube-api-access-gjvjt\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.827976 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7d151d7-5a60-43b0-bb19-b89cdb84f169-tmp-dir\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:00.827976 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.827826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rctqm\" (UniqueName: \"kubernetes.io/projected/c7d151d7-5a60-43b0-bb19-b89cdb84f169-kube-api-access-rctqm\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:00.929118 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7d151d7-5a60-43b0-bb19-b89cdb84f169-tmp-dir\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:00.929118 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rctqm\" (UniqueName: \"kubernetes.io/projected/c7d151d7-5a60-43b0-bb19-b89cdb84f169-kube-api-access-rctqm\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:00.929118 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.929361 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9678ae86-ae81-4ade-8164-8f491017c3d2-ca-trust-extracted\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.929361 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-certificates\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.929361 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9678ae86-ae81-4ade-8164-8f491017c3d2-image-registry-private-configuration\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.929507 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929383    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7d151d7-5a60-43b0-bb19-b89cdb84f169-tmp-dir\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:00.929507 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9678ae86-ae81-4ade-8164-8f491017c3d2-trusted-ca\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.929507 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-bound-sa-token\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.929507 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c7d151d7-5a60-43b0-bb19-b89cdb84f169-config-volume\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:00.929877 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:00.929501    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:00.929877 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:00.929524    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75555b558c-68ncn: secret "image-registry-tls" not found
Sep 10 20:59:00.929877 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:00.929877 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:00.929602    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls podName:9678ae86-ae81-4ade-8164-8f491017c3d2 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:01.429579836 +0000 UTC m=+35.109078672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls") pod "image-registry-75555b558c-68ncn" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2") : secret "image-registry-tls" not found
Sep 10 20:59:00.929877 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8df9efce-a745-452e-a40d-60a5463e4537-cert\") pod \"ingress-canary-6ntcm\" (UID: \"8df9efce-a745-452e-a40d-60a5463e4537\") " pod="openshift-ingress-canary/ingress-canary-6ntcm"
Sep 10 20:59:00.929877 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929669    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcwph\" (UniqueName: \"kubernetes.io/projected/8df9efce-a745-452e-a40d-60a5463e4537-kube-api-access-lcwph\") pod \"ingress-canary-6ntcm\" (UID: \"8df9efce-a745-452e-a40d-60a5463e4537\") " pod="openshift-ingress-canary/ingress-canary-6ntcm"
Sep 10 20:59:00.929877 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9678ae86-ae81-4ade-8164-8f491017c3d2-installation-pull-secrets\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.929877 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjvjt\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-kube-api-access-gjvjt\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.929877 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.929854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-certificates\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.930365 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:00.930034    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:00.930365 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:00.930108    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls podName:c7d151d7-5a60-43b0-bb19-b89cdb84f169 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:01.43008974 +0000 UTC m=+35.109588568 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls") pod "dns-default-h2hwr" (UID: "c7d151d7-5a60-43b0-bb19-b89cdb84f169") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:00.930365 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.930227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9678ae86-ae81-4ade-8164-8f491017c3d2-trusted-ca\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.930365 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.930304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c7d151d7-5a60-43b0-bb19-b89cdb84f169-config-volume\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:00.930579 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.930557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9678ae86-ae81-4ade-8164-8f491017c3d2-ca-trust-extracted\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.933466 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.933446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9678ae86-ae81-4ade-8164-8f491017c3d2-installation-pull-secrets\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.933553 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.933493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9678ae86-ae81-4ade-8164-8f491017c3d2-image-registry-private-configuration\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.933588 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.933572    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8df9efce-a745-452e-a40d-60a5463e4537-cert\") pod \"ingress-canary-6ntcm\" (UID: \"8df9efce-a745-452e-a40d-60a5463e4537\") " pod="openshift-ingress-canary/ingress-canary-6ntcm"
Sep 10 20:59:00.938639 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.938616    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcwph\" (UniqueName: \"kubernetes.io/projected/8df9efce-a745-452e-a40d-60a5463e4537-kube-api-access-lcwph\") pod \"ingress-canary-6ntcm\" (UID: \"8df9efce-a745-452e-a40d-60a5463e4537\") " pod="openshift-ingress-canary/ingress-canary-6ntcm"
Sep 10 20:59:00.939111 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.939088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-bound-sa-token\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.939611 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.939585    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjvjt\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-kube-api-access-gjvjt\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:00.944108 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.944087    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rctqm\" (UniqueName: \"kubernetes.io/projected/c7d151d7-5a60-43b0-bb19-b89cdb84f169-kube-api-access-rctqm\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:00.950140 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.950121    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:59:00.950219 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.950141    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:59:00.950219 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.950121    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:59:00.953044 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.953017    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 20:59:00.953158 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.953052    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 20:59:00.953158 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.953136    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wb7zl\""
Sep 10 20:59:00.953328 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.953030    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 20:59:00.953328 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.953205    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5khf6\""
Sep 10 20:59:00.953328 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:00.953245    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 20:59:01.032268 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:01.032235    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6ntcm"
Sep 10 20:59:01.211745 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:01.211668    2580 generic.go:356] "Generic (PLEG): container finished" podID="059d85a3-7029-4c96-81b6-9a1cefe6d1ec" containerID="827b85579aa37f2b98df4b11408852f8ea04dc0dea9d7912640388656e4d89ef" exitCode=0
Sep 10 20:59:01.211745 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:01.211707    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mxtq" event={"ID":"059d85a3-7029-4c96-81b6-9a1cefe6d1ec","Type":"ContainerDied","Data":"827b85579aa37f2b98df4b11408852f8ea04dc0dea9d7912640388656e4d89ef"}
Sep 10 20:59:01.326766 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:01.326561    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6ntcm"]
Sep 10 20:59:01.331309 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:59:01.331281    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8df9efce_a745_452e_a40d_60a5463e4537.slice/crio-8715e136057d88600e12e3ce2d9a348b29e11886fc99e2ad1fe40fd2faf3b8df WatchSource:0}: Error finding container 8715e136057d88600e12e3ce2d9a348b29e11886fc99e2ad1fe40fd2faf3b8df: Status 404 returned error can't find the container with id 8715e136057d88600e12e3ce2d9a348b29e11886fc99e2ad1fe40fd2faf3b8df
Sep 10 20:59:01.433809 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:01.433780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:01.433998 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:01.433843    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:01.433998 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:01.433932    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:01.433998 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:01.433969    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:01.433998 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:01.433983    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75555b558c-68ncn: secret "image-registry-tls" not found
Sep 10 20:59:01.433998 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:01.434000    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls podName:c7d151d7-5a60-43b0-bb19-b89cdb84f169 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:02.433981816 +0000 UTC m=+36.113480643 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls") pod "dns-default-h2hwr" (UID: "c7d151d7-5a60-43b0-bb19-b89cdb84f169") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:01.434229 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:01.434032    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls podName:9678ae86-ae81-4ade-8164-8f491017c3d2 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:02.434015676 +0000 UTC m=+36.113514503 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls") pod "image-registry-75555b558c-68ncn" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2") : secret "image-registry-tls" not found
Sep 10 20:59:01.567716 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:01.567633    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:02.215955 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:02.215918    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6ntcm" event={"ID":"8df9efce-a745-452e-a40d-60a5463e4537","Type":"ContainerStarted","Data":"8715e136057d88600e12e3ce2d9a348b29e11886fc99e2ad1fe40fd2faf3b8df"}
Sep 10 20:59:02.219411 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:02.219381    2580 generic.go:356] "Generic (PLEG): container finished" podID="059d85a3-7029-4c96-81b6-9a1cefe6d1ec" containerID="60174ff3e9c2a488ae0ff30005fc98c8322d5407bdde20f2d8397d463ceadad7" exitCode=0
Sep 10 20:59:02.219411 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:02.219428    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mxtq" event={"ID":"059d85a3-7029-4c96-81b6-9a1cefe6d1ec","Type":"ContainerDied","Data":"60174ff3e9c2a488ae0ff30005fc98c8322d5407bdde20f2d8397d463ceadad7"}
Sep 10 20:59:02.442583 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:02.442554    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:02.442742 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:02.442719    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:02.442809 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:02.442736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:02.442809 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:02.442746    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75555b558c-68ncn: secret "image-registry-tls" not found
Sep 10 20:59:02.442915 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:02.442808    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:02.442915 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:02.442867    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls podName:9678ae86-ae81-4ade-8164-8f491017c3d2 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:04.442846741 +0000 UTC m=+38.122345573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls") pod "image-registry-75555b558c-68ncn" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2") : secret "image-registry-tls" not found
Sep 10 20:59:02.442915 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:02.442889    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls podName:c7d151d7-5a60-43b0-bb19-b89cdb84f169 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:04.442879765 +0000 UTC m=+38.122378593 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls") pod "dns-default-h2hwr" (UID: "c7d151d7-5a60-43b0-bb19-b89cdb84f169") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:02.841868 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:02.841828    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:03.065357 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:03.065267    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:03.166075 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:03.165967    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:03.225776 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:03.225741    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8mxtq" event={"ID":"059d85a3-7029-4c96-81b6-9a1cefe6d1ec","Type":"ContainerStarted","Data":"3240011b6292184d59da7ebc721f01ab05e25e45dcb3d209e9d377947b9b60f9"}
Sep 10 20:59:03.251311 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:03.251254    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8mxtq" podStartSLOduration=8.806590746 podStartE2EDuration="36.251236931s" totalImagesPullingTime="27.444646185s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:03.250148321 +0000 UTC m=+36.929647172" watchObservedRunningTime="2026-09-10 20:59:03.251236931 +0000 UTC m=+36.930735780"
Sep 10 20:59:03.859533 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:03.859505    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:03.960664 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:03.960481    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:04.229634 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:04.229597    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6ntcm" event={"ID":"8df9efce-a745-452e-a40d-60a5463e4537","Type":"ContainerStarted","Data":"9a9c2160b7755c9d2580d05d17873d2f8e9053a409f8eede8e3079b8ea52b541"}
Sep 10 20:59:04.252653 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:04.252612    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6ntcm" podStartSLOduration=1.806940064 podStartE2EDuration="4.252595902s" totalImagesPullingTime="2.445655838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:04.252570016 +0000 UTC m=+37.932068865" watchObservedRunningTime="2026-09-10 20:59:04.252595902 +0000 UTC m=+37.932094748"
Sep 10 20:59:04.278369 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:04.278336    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:04.377243 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:04.377211    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:04.459302 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:04.459272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:04.459407 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:04.459326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:04.459479 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:04.459427    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:04.459479 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:04.459434    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:04.459593 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:04.459503    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls podName:c7d151d7-5a60-43b0-bb19-b89cdb84f169 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:08.4594802 +0000 UTC m=+42.138979042 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls") pod "dns-default-h2hwr" (UID: "c7d151d7-5a60-43b0-bb19-b89cdb84f169") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:04.459593 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:04.459442    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75555b558c-68ncn: secret "image-registry-tls" not found
Sep 10 20:59:04.459593 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:04.459563    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls podName:9678ae86-ae81-4ade-8164-8f491017c3d2 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:08.459552545 +0000 UTC m=+42.139051374 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls") pod "image-registry-75555b558c-68ncn" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2") : secret "image-registry-tls" not found
Sep 10 20:59:06.436341 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:06.436312    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:06.511924 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:06.511885    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:08.486573 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:08.486534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:08.486933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:08.486627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:08.486933 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:08.486697    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:08.486933 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:08.486713    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75555b558c-68ncn: secret "image-registry-tls" not found
Sep 10 20:59:08.486933 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:08.486770    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls podName:9678ae86-ae81-4ade-8164-8f491017c3d2 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:16.486756456 +0000 UTC m=+50.166255288 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls") pod "image-registry-75555b558c-68ncn" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2") : secret "image-registry-tls" not found
Sep 10 20:59:08.486933 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:08.486780    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:08.486933 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:08.486832    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls podName:c7d151d7-5a60-43b0-bb19-b89cdb84f169 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:16.486818324 +0000 UTC m=+50.166317152 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls") pod "dns-default-h2hwr" (UID: "c7d151d7-5a60-43b0-bb19-b89cdb84f169") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:11.966811 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:11.966782    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:12.064962 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:12.064925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:12.164675 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:12.164644    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:13.163027 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:13.162949    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:16.339591 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:16.339556    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:16.398758 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:16.398729    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:16.548400 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:16.548361    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:16.548580 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:16.548429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:16.548580 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:16.548511    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:16.548580 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:16.548530    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75555b558c-68ncn: secret "image-registry-tls" not found
Sep 10 20:59:16.548580 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:16.548552    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:16.548744 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:16.548588    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls podName:9678ae86-ae81-4ade-8164-8f491017c3d2 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:32.548571675 +0000 UTC m=+66.228070503 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls") pod "image-registry-75555b558c-68ncn" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2") : secret "image-registry-tls" not found
Sep 10 20:59:16.548744 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:16.548613    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls podName:c7d151d7-5a60-43b0-bb19-b89cdb84f169 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:32.548594149 +0000 UTC m=+66.228092998 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls") pod "dns-default-h2hwr" (UID: "c7d151d7-5a60-43b0-bb19-b89cdb84f169") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:22.244775 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:22.244733    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:22.343837 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:22.343804    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:22.410773 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:22.410737    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:23.123964 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:23.123926    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:23.180338 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:23.180308    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:23.241179 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:23.241147    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:25.209239 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:25.209212    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2qkd6"
Sep 10 20:59:25.374965 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:25.374930    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:25.442578 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:25.442546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:26.058275 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:26.058242    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:32.560153 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.560114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 20:59:32.560638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.560164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 20:59:32.560638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.560188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:59:32.560638 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.560223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 20:59:32.560638 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:32.560303    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:32.560638 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:32.560328    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75555b558c-68ncn: secret "image-registry-tls" not found
Sep 10 20:59:32.560638 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:32.560332    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:32.560638 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:32.560390    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls podName:c7d151d7-5a60-43b0-bb19-b89cdb84f169 nodeName:}" failed. No retries permitted until 2026-09-10 21:00:04.560375145 +0000 UTC m=+98.239873973 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls") pod "dns-default-h2hwr" (UID: "c7d151d7-5a60-43b0-bb19-b89cdb84f169") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:32.560638 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:32.560405    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls podName:9678ae86-ae81-4ade-8164-8f491017c3d2 nodeName:}" failed. No retries permitted until 2026-09-10 21:00:04.560398949 +0000 UTC m=+98.239897777 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls") pod "image-registry-75555b558c-68ncn" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2") : secret "image-registry-tls" not found
Sep 10 20:59:32.563105 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.563083    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 20:59:32.563201 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.563175    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 20:59:32.571351 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:32.571332    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 20:59:32.571486 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:32.571379    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs podName:1bb0820e-193e-416f-a950-5eedb01fc196 nodeName:}" failed. No retries permitted until 2026-09-10 21:00:36.571362863 +0000 UTC m=+130.250861694 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs") pod "network-metrics-daemon-dkc7w" (UID: "1bb0820e-193e-416f-a950-5eedb01fc196") : secret "metrics-daemon-secret" not found
Sep 10 20:59:32.575011 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.574987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0819e2f5-b35c-4836-8d5e-939990ad6009-original-pull-secret\") pod \"global-pull-secret-syncer-k7pdz\" (UID: \"0819e2f5-b35c-4836-8d5e-939990ad6009\") " pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:59:32.661012 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.660982    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl4gb\" (UniqueName: \"kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb\") pod \"network-check-target-pcwdg\" (UID: \"b52aa986-0758-4fd0-a230-498beba6b387\") " pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:59:32.664166 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.664148    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 20:59:32.674284 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.674267    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 20:59:32.684592 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.684572    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl4gb\" (UniqueName: \"kubernetes.io/projected/b52aa986-0758-4fd0-a230-498beba6b387-kube-api-access-sl4gb\") pod \"network-check-target-pcwdg\" (UID: \"b52aa986-0758-4fd0-a230-498beba6b387\") " pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:59:32.762610 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.762588    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wb7zl\""
Sep 10 20:59:32.769965 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.769949    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:59:32.770051 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:32.769968    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k7pdz"
Sep 10 20:59:33.075573 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:33.075551    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pcwdg"]
Sep 10 20:59:33.077017 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:59:33.076990    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb52aa986_0758_4fd0_a230_498beba6b387.slice/crio-7501bbb16221af5ecf362f458b2f0f549a3f3b6f78698e41e674c26c798266cf WatchSource:0}: Error finding container 7501bbb16221af5ecf362f458b2f0f549a3f3b6f78698e41e674c26c798266cf: Status 404 returned error can't find the container with id 7501bbb16221af5ecf362f458b2f0f549a3f3b6f78698e41e674c26c798266cf
Sep 10 20:59:33.104287 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:33.104266    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k7pdz"]
Sep 10 20:59:33.107133 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:59:33.107111    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0819e2f5_b35c_4836_8d5e_939990ad6009.slice/crio-d31483e653a0ae2bba31c4f6051791182d8ff403bf34a0446bf64f9b21cae662 WatchSource:0}: Error finding container d31483e653a0ae2bba31c4f6051791182d8ff403bf34a0446bf64f9b21cae662: Status 404 returned error can't find the container with id d31483e653a0ae2bba31c4f6051791182d8ff403bf34a0446bf64f9b21cae662
Sep 10 20:59:33.297650 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:33.297618    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k7pdz" event={"ID":"0819e2f5-b35c-4836-8d5e-939990ad6009","Type":"ContainerStarted","Data":"d31483e653a0ae2bba31c4f6051791182d8ff403bf34a0446bf64f9b21cae662"}
Sep 10 20:59:33.298780 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:33.298753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pcwdg" event={"ID":"b52aa986-0758-4fd0-a230-498beba6b387","Type":"ContainerStarted","Data":"7501bbb16221af5ecf362f458b2f0f549a3f3b6f78698e41e674c26c798266cf"}
Sep 10 20:59:34.424476 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:34.424388    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:35.422734 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:35.422690    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:35.483184 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:35.483151    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:35.544898 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:35.544867    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:35.793925 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:35.793892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:35.860873 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:35.860850    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:35.963188 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:35.963157    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:36.187482 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:36.187408    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:36.249328 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:36.249295    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:36.304719 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:36.304682    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:38.313404 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:38.313364    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k7pdz" event={"ID":"0819e2f5-b35c-4836-8d5e-939990ad6009","Type":"ContainerStarted","Data":"8e70900da7cafb653b7940eccd5e846ab338e99570c175bb66e3c6706a858ee1"}
Sep 10 20:59:38.314893 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:38.314866    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pcwdg" event={"ID":"b52aa986-0758-4fd0-a230-498beba6b387","Type":"ContainerStarted","Data":"b0a6c1d06b4673c155444e847efaee23690eb7b943b3397fe8c361886e3a2e99"}
Sep 10 20:59:38.315031 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:38.314989    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 20:59:38.335703 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:38.335595    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k7pdz" podStartSLOduration=67.061390262 podStartE2EDuration="1m11.335561949s" totalImagesPullingTime="4.274171687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:38.335542367 +0000 UTC m=+72.015041219" watchObservedRunningTime="2026-09-10 20:59:38.335561949 +0000 UTC m=+72.015060788"
Sep 10 20:59:38.357112 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:38.357034    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pcwdg" podStartSLOduration=67.057300902 podStartE2EDuration="1m11.357012755s" totalImagesPullingTime="4.299711853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:38.356038634 +0000 UTC m=+72.035537484" watchObservedRunningTime="2026-09-10 20:59:38.357012755 +0000 UTC m=+72.036511605"
Sep 10 20:59:38.953944 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:38.953916    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-p9rh5"]
Sep 10 20:59:38.956756 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:38.956739    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-p9rh5"
Sep 10 20:59:38.959621 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:38.959601    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-dccds\""
Sep 10 20:59:38.972104 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:38.972081    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-p9rh5"]
Sep 10 20:59:39.012183 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:39.012153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hpgm\" (UniqueName: \"kubernetes.io/projected/eb936e38-31c0-4211-8c7e-8bcc3c2427ed-kube-api-access-9hpgm\") pod \"network-check-source-6f8b768455-p9rh5\" (UID: \"eb936e38-31c0-4211-8c7e-8bcc3c2427ed\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-p9rh5"
Sep 10 20:59:39.112773 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:39.112736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hpgm\" (UniqueName: \"kubernetes.io/projected/eb936e38-31c0-4211-8c7e-8bcc3c2427ed-kube-api-access-9hpgm\") pod \"network-check-source-6f8b768455-p9rh5\" (UID: \"eb936e38-31c0-4211-8c7e-8bcc3c2427ed\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-p9rh5"
Sep 10 20:59:39.123155 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:39.123130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hpgm\" (UniqueName: \"kubernetes.io/projected/eb936e38-31c0-4211-8c7e-8bcc3c2427ed-kube-api-access-9hpgm\") pod \"network-check-source-6f8b768455-p9rh5\" (UID: \"eb936e38-31c0-4211-8c7e-8bcc3c2427ed\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-p9rh5"
Sep 10 20:59:39.168370 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:39.168338    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:39.265097 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:39.264994    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-p9rh5"
Sep 10 20:59:39.269884 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:39.269864    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:39.533429 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:39.533354    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-p9rh5"]
Sep 10 20:59:39.536011 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:59:39.535977    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb936e38_31c0_4211_8c7e_8bcc3c2427ed.slice/crio-aa45dbc4305a41f9a2b739631a0719e9f6e79b073471d52d467c8e5c2c6992c8 WatchSource:0}: Error finding container aa45dbc4305a41f9a2b739631a0719e9f6e79b073471d52d467c8e5c2c6992c8: Status 404 returned error can't find the container with id aa45dbc4305a41f9a2b739631a0719e9f6e79b073471d52d467c8e5c2c6992c8
Sep 10 20:59:40.321519 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:40.321483    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-p9rh5" event={"ID":"eb936e38-31c0-4211-8c7e-8bcc3c2427ed","Type":"ContainerStarted","Data":"f46dbdaff7c74bd58f6768f772c33726f329db755d8c1b2e60d2235ddbd57b83"}
Sep 10 20:59:40.321700 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:40.321537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-p9rh5" event={"ID":"eb936e38-31c0-4211-8c7e-8bcc3c2427ed","Type":"ContainerStarted","Data":"aa45dbc4305a41f9a2b739631a0719e9f6e79b073471d52d467c8e5c2c6992c8"}
Sep 10 20:59:40.338461 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:40.338395    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-p9rh5" podStartSLOduration=2.338379028 podStartE2EDuration="2.338379028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:40.337870283 +0000 UTC m=+74.017369143" watchObservedRunningTime="2026-09-10 20:59:40.338379028 +0000 UTC m=+74.017877880"
Sep 10 20:59:40.727386 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:40.727304    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:42.020472 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:42.020442    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:42.121026 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:42.120994    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:43.594868 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:43.594834    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:43.686149 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:43.686119    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:43.747263 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:43.747232    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:44.950271 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:44.950239    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:45.008418 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.008368    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:45.649771 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.649733    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:45.701141 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.701105    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vqzv4"]
Sep 10 20:59:45.704425 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.704396    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:45.707241 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.707218    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 20:59:45.707390 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.707256    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 20:59:45.707390 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.707288    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-89wjp\""
Sep 10 20:59:45.707390 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.707330    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 20:59:45.708027 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.708012    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 20:59:45.710837 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.710790    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:45.716316 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.716295    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vqzv4"]
Sep 10 20:59:45.763137 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.763095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b92e8363-eb18-4e60-8bce-9f5286beccfa-crio-socket\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:45.763137 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.763139    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:45.763348 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.763197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:45.763348 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.763237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b92e8363-eb18-4e60-8bce-9f5286beccfa-data-volume\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:45.763348 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.763254    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d68ch\" (UniqueName: \"kubernetes.io/projected/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-api-access-d68ch\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:45.863973 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.863929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b92e8363-eb18-4e60-8bce-9f5286beccfa-crio-socket\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:45.863973 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.863967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:45.864246 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.863985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:45.864246 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.864006    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b92e8363-eb18-4e60-8bce-9f5286beccfa-data-volume\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:45.864246 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.864021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d68ch\" (UniqueName: \"kubernetes.io/projected/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-api-access-d68ch\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:45.864246 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.864053    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b92e8363-eb18-4e60-8bce-9f5286beccfa-crio-socket\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:45.864246 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:45.864106    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:45.864246 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:45.864144    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:45.864246 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:45.864206    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm podName:b92e8363-eb18-4e60-8bce-9f5286beccfa nodeName:}" failed. No retries permitted until 2026-09-10 20:59:46.364183201 +0000 UTC m=+80.043682046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vqzv4" (UID: "b92e8363-eb18-4e60-8bce-9f5286beccfa") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:45.864246 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:45.864227    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls podName:b92e8363-eb18-4e60-8bce-9f5286beccfa nodeName:}" failed. No retries permitted until 2026-09-10 20:59:46.364217044 +0000 UTC m=+80.043715881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqzv4" (UID: "b92e8363-eb18-4e60-8bce-9f5286beccfa") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:45.864506 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.864394    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b92e8363-eb18-4e60-8bce-9f5286beccfa-data-volume\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:45.875086 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:45.875047    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d68ch\" (UniqueName: \"kubernetes.io/projected/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-api-access-d68ch\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:46.367013 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:46.366976    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:46.367013 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:46.367010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:46.367446 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:46.367112    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:46.367446 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:46.367128    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:46.367446 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:46.367186    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm podName:b92e8363-eb18-4e60-8bce-9f5286beccfa nodeName:}" failed. No retries permitted until 2026-09-10 20:59:47.36716651 +0000 UTC m=+81.046665346 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vqzv4" (UID: "b92e8363-eb18-4e60-8bce-9f5286beccfa") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:46.367446 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:46.367201    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls podName:b92e8363-eb18-4e60-8bce-9f5286beccfa nodeName:}" failed. No retries permitted until 2026-09-10 20:59:47.36719535 +0000 UTC m=+81.046694177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqzv4" (UID: "b92e8363-eb18-4e60-8bce-9f5286beccfa") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:47.207209 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.207171    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:47.322803 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.322764    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-rr2d6"]
Sep 10 20:59:47.325609 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.325585    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-rr2d6"
Sep 10 20:59:47.326127 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.326104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:47.329182 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.329155    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 20:59:47.329280 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.329269    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-gkxjq\""
Sep 10 20:59:47.329926 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.329908    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 20:59:47.330004 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.329932    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 20:59:47.330368 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.330350    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 20:59:47.339874 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.339850    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-rr2d6"]
Sep 10 20:59:47.374142 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.374114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:47.374142 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.374145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:47.374475 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.374198    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bce12e11-2a9d-4056-bac1-4a22be6fc573-signing-key\") pod \"service-ca-568869fdc5-rr2d6\" (UID: \"bce12e11-2a9d-4056-bac1-4a22be6fc573\") " pod="openshift-service-ca/service-ca-568869fdc5-rr2d6"
Sep 10 20:59:47.374475 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.374219    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptl5n\" (UniqueName: \"kubernetes.io/projected/bce12e11-2a9d-4056-bac1-4a22be6fc573-kube-api-access-ptl5n\") pod \"service-ca-568869fdc5-rr2d6\" (UID: \"bce12e11-2a9d-4056-bac1-4a22be6fc573\") " pod="openshift-service-ca/service-ca-568869fdc5-rr2d6"
Sep 10 20:59:47.374475 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.374239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bce12e11-2a9d-4056-bac1-4a22be6fc573-signing-cabundle\") pod \"service-ca-568869fdc5-rr2d6\" (UID: \"bce12e11-2a9d-4056-bac1-4a22be6fc573\") " pod="openshift-service-ca/service-ca-568869fdc5-rr2d6"
Sep 10 20:59:47.374475 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:47.374220    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:47.374475 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:47.374290    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:47.374475 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:47.374350    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls podName:b92e8363-eb18-4e60-8bce-9f5286beccfa nodeName:}" failed. No retries permitted until 2026-09-10 20:59:49.374331329 +0000 UTC m=+83.053830171 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqzv4" (UID: "b92e8363-eb18-4e60-8bce-9f5286beccfa") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:47.374475 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:47.374364    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm podName:b92e8363-eb18-4e60-8bce-9f5286beccfa nodeName:}" failed. No retries permitted until 2026-09-10 20:59:49.374357522 +0000 UTC m=+83.053856350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vqzv4" (UID: "b92e8363-eb18-4e60-8bce-9f5286beccfa") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:47.402095 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.402070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:47.474933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.474863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bce12e11-2a9d-4056-bac1-4a22be6fc573-signing-key\") pod \"service-ca-568869fdc5-rr2d6\" (UID: \"bce12e11-2a9d-4056-bac1-4a22be6fc573\") " pod="openshift-service-ca/service-ca-568869fdc5-rr2d6"
Sep 10 20:59:47.474933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.474900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptl5n\" (UniqueName: \"kubernetes.io/projected/bce12e11-2a9d-4056-bac1-4a22be6fc573-kube-api-access-ptl5n\") pod \"service-ca-568869fdc5-rr2d6\" (UID: \"bce12e11-2a9d-4056-bac1-4a22be6fc573\") " pod="openshift-service-ca/service-ca-568869fdc5-rr2d6"
Sep 10 20:59:47.474933 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.474928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bce12e11-2a9d-4056-bac1-4a22be6fc573-signing-cabundle\") pod \"service-ca-568869fdc5-rr2d6\" (UID: \"bce12e11-2a9d-4056-bac1-4a22be6fc573\") " pod="openshift-service-ca/service-ca-568869fdc5-rr2d6"
Sep 10 20:59:47.475674 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.475648    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bce12e11-2a9d-4056-bac1-4a22be6fc573-signing-cabundle\") pod \"service-ca-568869fdc5-rr2d6\" (UID: \"bce12e11-2a9d-4056-bac1-4a22be6fc573\") " pod="openshift-service-ca/service-ca-568869fdc5-rr2d6"
Sep 10 20:59:47.477311 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.477295    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bce12e11-2a9d-4056-bac1-4a22be6fc573-signing-key\") pod \"service-ca-568869fdc5-rr2d6\" (UID: \"bce12e11-2a9d-4056-bac1-4a22be6fc573\") " pod="openshift-service-ca/service-ca-568869fdc5-rr2d6"
Sep 10 20:59:47.492071 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.492042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptl5n\" (UniqueName: \"kubernetes.io/projected/bce12e11-2a9d-4056-bac1-4a22be6fc573-kube-api-access-ptl5n\") pod \"service-ca-568869fdc5-rr2d6\" (UID: \"bce12e11-2a9d-4056-bac1-4a22be6fc573\") " pod="openshift-service-ca/service-ca-568869fdc5-rr2d6"
Sep 10 20:59:47.634178 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.634137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-rr2d6"
Sep 10 20:59:47.911658 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:47.911582    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-rr2d6"]
Sep 10 20:59:47.915190 ip-10-0-135-142 kubenswrapper[2580]: W0910 20:59:47.915159    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbce12e11_2a9d_4056_bac1_4a22be6fc573.slice/crio-2d4dbcb28ab4e02e641c616280025e5933e281b15822b53b22301812b4f25a5b WatchSource:0}: Error finding container 2d4dbcb28ab4e02e641c616280025e5933e281b15822b53b22301812b4f25a5b: Status 404 returned error can't find the container with id 2d4dbcb28ab4e02e641c616280025e5933e281b15822b53b22301812b4f25a5b
Sep 10 20:59:48.341241 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:48.341210    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-rr2d6" event={"ID":"bce12e11-2a9d-4056-bac1-4a22be6fc573","Type":"ContainerStarted","Data":"2d4dbcb28ab4e02e641c616280025e5933e281b15822b53b22301812b4f25a5b"}
Sep 10 20:59:48.674264 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:48.674191    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xhglt_ed9cd22c-bd90-473a-9814-eea2735c3cf0/dns-node-resolver/0.log"
Sep 10 20:59:49.275188 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:49.275157    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4gz27_ea3e1564-e8e9-47db-a5e3-86002afeaeab/node-ca/0.log"
Sep 10 20:59:49.389601 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:49.389562    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:49.389800 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:49.389608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:49.389800 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:49.389666    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:49.389800 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:49.389751    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm podName:b92e8363-eb18-4e60-8bce-9f5286beccfa nodeName:}" failed. No retries permitted until 2026-09-10 20:59:53.389727858 +0000 UTC m=+87.069226687 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vqzv4" (UID: "b92e8363-eb18-4e60-8bce-9f5286beccfa") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:49.389800 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:49.389754    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:49.390025 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:49.389804    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls podName:b92e8363-eb18-4e60-8bce-9f5286beccfa nodeName:}" failed. No retries permitted until 2026-09-10 20:59:53.38978788 +0000 UTC m=+87.069286709 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqzv4" (UID: "b92e8363-eb18-4e60-8bce-9f5286beccfa") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:50.073395 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:50.073361    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6ntcm_8df9efce-a745-452e-a40d-60a5463e4537/serve-healthcheck-canary/0.log"
Sep 10 20:59:50.095669 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:50.095637    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:50.347503 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:50.347426    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-rr2d6" event={"ID":"bce12e11-2a9d-4056-bac1-4a22be6fc573","Type":"ContainerStarted","Data":"8d939905c2c1f1cf1a306602147a6d602fd597178c6000b8da533fbbb9d466a5"}
Sep 10 20:59:50.371186 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:50.371019    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-rr2d6" podStartSLOduration=1.537271821 podStartE2EDuration="3.370985134s" totalImagesPullingTime="1.833713313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:50.369095259 +0000 UTC m=+84.048594666" watchObservedRunningTime="2026-09-10 20:59:50.370985134 +0000 UTC m=+84.050484066"
Sep 10 20:59:51.381970 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:51.381936    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:51.519696 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:51.519656    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:51.622874 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:51.622842    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:51.720794 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:51.720699    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:51.858255 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:51.858224    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:51.949627 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:51.949594    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:52.101314 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:52.101280    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:53.279965 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:53.279930    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:53.427950 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:53.427906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:53.428173 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:53.427958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 20:59:53.428173 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:53.428032    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:53.428173 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:53.428126    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:53.428173 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:53.428130    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm podName:b92e8363-eb18-4e60-8bce-9f5286beccfa nodeName:}" failed. No retries permitted until 2026-09-10 21:00:01.428106278 +0000 UTC m=+95.107605123 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vqzv4" (UID: "b92e8363-eb18-4e60-8bce-9f5286beccfa") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:53.428173 ip-10-0-135-142 kubenswrapper[2580]: E0910 20:59:53.428175    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls podName:b92e8363-eb18-4e60-8bce-9f5286beccfa nodeName:}" failed. No retries permitted until 2026-09-10 21:00:01.428163715 +0000 UTC m=+95.107662557 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqzv4" (UID: "b92e8363-eb18-4e60-8bce-9f5286beccfa") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:53.476565 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:53.476525    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:53.681720 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:53.681686    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:53.888074 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:53.888031    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:54.082730 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:54.082697    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:54.641861 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:54.641828    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:54.742651 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:54.742621    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:54.878625 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:54.878592    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:55.187028 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:55.186996    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:55.481327 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:55.481243    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:55.680540 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:55.680505    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:56.082968 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:56.082938    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:56.183209 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:56.183179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:56.282779 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:56.282751    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:56.384783 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:56.384701    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:56.693714 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:56.693631    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:56.794223 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:56.794189    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:57.085195 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:57.085162    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:57.677429 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:57.677401    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:58.177785 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:58.177752    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:58.281909 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:58.281876    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:58.476526 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:58.476446    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:58.674274 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:58.674239    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:59.076438 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:59.076404    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 20:59:59.675685 ip-10-0-135-142 kubenswrapper[2580]: I0910 20:59:59.675650    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:00.283200 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:00.283162    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:00.481137 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:00.481101    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:00.687869 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:00.687841    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:01.077117 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:01.077088    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:01.409236 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:01.409150    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:01.493794 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:01.493762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 21:00:01.493794 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:01.493796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 21:00:01.494034 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:00:01.493974    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:00:01.494034 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:00:01.494026    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls podName:b92e8363-eb18-4e60-8bce-9f5286beccfa nodeName:}" failed. No retries permitted until 2026-09-10 21:00:17.494012306 +0000 UTC m=+111.173511146 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vqzv4" (UID: "b92e8363-eb18-4e60-8bce-9f5286beccfa") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:00:01.494387 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:01.494363    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b92e8363-eb18-4e60-8bce-9f5286beccfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 21:00:01.508617 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:01.508593    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:01.609177 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:01.609145    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:02.055902 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:02.055872    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:02.350034 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:02.349948    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:04.619835 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:04.619791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 21:00:04.620288 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:04.619868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 21:00:04.622153 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:04.622130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7d151d7-5a60-43b0-bb19-b89cdb84f169-metrics-tls\") pod \"dns-default-h2hwr\" (UID: \"c7d151d7-5a60-43b0-bb19-b89cdb84f169\") " pod="openshift-dns/dns-default-h2hwr"
Sep 10 21:00:04.622273 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:04.622253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls\") pod \"image-registry-75555b558c-68ncn\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") " pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 21:00:04.628263 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:04.628241    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7j9n7\""
Sep 10 21:00:04.635947 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:04.635930    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h2hwr"
Sep 10 21:00:04.895927 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:04.895852    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-k4c27\""
Sep 10 21:00:04.904125 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:04.904102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 21:00:04.909136 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:04.909110    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h2hwr"]
Sep 10 21:00:04.912635 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:00:04.912611    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7d151d7_5a60_43b0_bb19_b89cdb84f169.slice/crio-3d0f98e5633f1c054729fd455d5387514053eb2436dd2fe6c7364c92fca09c8b WatchSource:0}: Error finding container 3d0f98e5633f1c054729fd455d5387514053eb2436dd2fe6c7364c92fca09c8b: Status 404 returned error can't find the container with id 3d0f98e5633f1c054729fd455d5387514053eb2436dd2fe6c7364c92fca09c8b
Sep 10 21:00:05.182570 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:05.182498    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75555b558c-68ncn"]
Sep 10 21:00:05.185753 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:00:05.185726    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9678ae86_ae81_4ade_8164_8f491017c3d2.slice/crio-bbc59eafac1e531917565f1ad91537ab83db31815a4487c49dab1334ef35346a WatchSource:0}: Error finding container bbc59eafac1e531917565f1ad91537ab83db31815a4487c49dab1334ef35346a: Status 404 returned error can't find the container with id bbc59eafac1e531917565f1ad91537ab83db31815a4487c49dab1334ef35346a
Sep 10 21:00:05.387916 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:05.387876    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75555b558c-68ncn" event={"ID":"9678ae86-ae81-4ade-8164-8f491017c3d2","Type":"ContainerStarted","Data":"650b84013224e5d7787aa4498e1936218794050b2e70ea646b06881c8279474e"}
Sep 10 21:00:05.388148 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:05.387938    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75555b558c-68ncn" event={"ID":"9678ae86-ae81-4ade-8164-8f491017c3d2","Type":"ContainerStarted","Data":"bbc59eafac1e531917565f1ad91537ab83db31815a4487c49dab1334ef35346a"}
Sep 10 21:00:05.388148 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:05.387961    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 21:00:05.389146 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:05.389122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h2hwr" event={"ID":"c7d151d7-5a60-43b0-bb19-b89cdb84f169","Type":"ContainerStarted","Data":"3d0f98e5633f1c054729fd455d5387514053eb2436dd2fe6c7364c92fca09c8b"}
Sep 10 21:00:05.409467 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:05.409420    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75555b558c-68ncn" podStartSLOduration=98.409406051 podStartE2EDuration="1m38.409406051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:05.40834615 +0000 UTC m=+99.087845013" watchObservedRunningTime="2026-09-10 21:00:05.409406051 +0000 UTC m=+99.088904898"
Sep 10 21:00:05.617004 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:05.616969    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:05.672651 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:05.672620    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:05.738036 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:05.738002    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:06.125903 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:06.125871    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:06.225179 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:06.225142    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:06.524032 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:06.523945    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:06.622240 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:06.622203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:06.723977 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:06.723918    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:07.165477 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:07.165439    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:07.367509 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:07.367479    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:07.840536 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:07.840512    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:07.996330 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:07.996276    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-k9dkd"]
Sep 10 21:00:08.009083 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.009045    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-k9dkd"
Sep 10 21:00:08.011604 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.011584    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-6kls8\""
Sep 10 21:00:08.011731 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.011607    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 21:00:08.011949 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.011933    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 21:00:08.015365 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.015343    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-k9dkd"]
Sep 10 21:00:08.092013 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.091980    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:08.149815 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.149782    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2qg7\" (UniqueName: \"kubernetes.io/projected/1d790ed2-e8f5-4eea-abd2-2af131a17bd8-kube-api-access-v2qg7\") pod \"downloads-76f99584f4-k9dkd\" (UID: \"1d790ed2-e8f5-4eea-abd2-2af131a17bd8\") " pod="openshift-console/downloads-76f99584f4-k9dkd"
Sep 10 21:00:08.194464 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.194427    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:08.250195 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.250108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2qg7\" (UniqueName: \"kubernetes.io/projected/1d790ed2-e8f5-4eea-abd2-2af131a17bd8-kube-api-access-v2qg7\") pod \"downloads-76f99584f4-k9dkd\" (UID: \"1d790ed2-e8f5-4eea-abd2-2af131a17bd8\") " pod="openshift-console/downloads-76f99584f4-k9dkd"
Sep 10 21:00:08.259775 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.259750    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2qg7\" (UniqueName: \"kubernetes.io/projected/1d790ed2-e8f5-4eea-abd2-2af131a17bd8-kube-api-access-v2qg7\") pod \"downloads-76f99584f4-k9dkd\" (UID: \"1d790ed2-e8f5-4eea-abd2-2af131a17bd8\") " pod="openshift-console/downloads-76f99584f4-k9dkd"
Sep 10 21:00:08.295157 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.295126    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:08.316977 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.316951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-k9dkd"
Sep 10 21:00:08.407157 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.407038    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h2hwr" event={"ID":"c7d151d7-5a60-43b0-bb19-b89cdb84f169","Type":"ContainerStarted","Data":"7a4934cbf41a7bba24193a7d022cf8a6bd383236eaf16b5f09128de8445b131d"}
Sep 10 21:00:08.407157 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.407125    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h2hwr" event={"ID":"c7d151d7-5a60-43b0-bb19-b89cdb84f169","Type":"ContainerStarted","Data":"01fddee7e8c1d4b0cfc85eab27fadc1f00268a04999c310471123554020a983b"}
Sep 10 21:00:08.407157 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.407158    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h2hwr"
Sep 10 21:00:08.428980 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.428793    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-h2hwr" podStartSLOduration=65.596744823 podStartE2EDuration="1m8.428742s" totalImagesPullingTime="2.831997177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:08.427622108 +0000 UTC m=+102.107120957" watchObservedRunningTime="2026-09-10 21:00:08.428742 +0000 UTC m=+102.108240851"
Sep 10 21:00:08.616354 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.616263    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-k9dkd"]
Sep 10 21:00:08.623009 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:00:08.622941    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d790ed2_e8f5_4eea_abd2_2af131a17bd8.slice/crio-fea00732aed5fc3d5163faeed0685fe79c728aceabe3d671a80cfa7c11ea37c9 WatchSource:0}: Error finding container fea00732aed5fc3d5163faeed0685fe79c728aceabe3d671a80cfa7c11ea37c9: Status 404 returned error can't find the container with id fea00732aed5fc3d5163faeed0685fe79c728aceabe3d671a80cfa7c11ea37c9
Sep 10 21:00:08.731506 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.731433    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:08.841303 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:08.841237    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:09.321009 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:09.320979    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pcwdg"
Sep 10 21:00:09.327414 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:09.327384    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:09.412894 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:09.412851    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-k9dkd" event={"ID":"1d790ed2-e8f5-4eea-abd2-2af131a17bd8","Type":"ContainerStarted","Data":"fea00732aed5fc3d5163faeed0685fe79c728aceabe3d671a80cfa7c11ea37c9"}
Sep 10 21:00:09.439160 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:09.439126    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:09.528078 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:09.528015    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:14.807129 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:14.807092    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84c98888f8-4jvjd"]
Sep 10 21:00:14.811841 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:14.811821    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:14.814574 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:14.814540    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 21:00:14.814727 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:14.814704    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 21:00:14.815707 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:14.815687    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 21:00:14.815810 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:14.815756    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 21:00:14.815884 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:14.815870    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 21:00:14.815994 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:14.815979    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7tkmb\""
Sep 10 21:00:14.822797 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:14.822777    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84c98888f8-4jvjd"]
Sep 10 21:00:15.005651 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.005607    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snd2l\" (UniqueName: \"kubernetes.io/projected/120bc529-f002-42fc-8ac6-653eacb78477-kube-api-access-snd2l\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.005833 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.005701    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/120bc529-f002-42fc-8ac6-653eacb78477-console-serving-cert\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.005833 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.005725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/120bc529-f002-42fc-8ac6-653eacb78477-console-oauth-config\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.005833 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.005753    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-console-config\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.005833 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.005784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-service-ca\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.006010 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.005849    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-oauth-serving-cert\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.107125 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.107028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/120bc529-f002-42fc-8ac6-653eacb78477-console-serving-cert\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.107285 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.107179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/120bc529-f002-42fc-8ac6-653eacb78477-console-oauth-config\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.107285 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.107220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-console-config\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.107285 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.107251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-service-ca\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.107452 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.107291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-oauth-serving-cert\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.107452 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.107336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snd2l\" (UniqueName: \"kubernetes.io/projected/120bc529-f002-42fc-8ac6-653eacb78477-kube-api-access-snd2l\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.108022 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.107986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-console-config\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.108170 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.108049    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-service-ca\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.108170 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.108097    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-oauth-serving-cert\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.110190 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.110164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/120bc529-f002-42fc-8ac6-653eacb78477-console-oauth-config\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.110316 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.110188    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/120bc529-f002-42fc-8ac6-653eacb78477-console-serving-cert\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.135424 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.135398    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snd2l\" (UniqueName: \"kubernetes.io/projected/120bc529-f002-42fc-8ac6-653eacb78477-kube-api-access-snd2l\") pod \"console-84c98888f8-4jvjd\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") " pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.163941 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.163902    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:15.372660 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.372587    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94"]
Sep 10 21:00:15.377215 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.377192    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94"
Sep 10 21:00:15.380823 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.380800    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 21:00:15.380946 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.380924    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-lv2l2\""
Sep 10 21:00:15.382869 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.382817    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:15.390696 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.390671    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94"]
Sep 10 21:00:15.409947 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.409917    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fd6c8a79-0fec-4e70-a85c-6613f42ad6b9-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-pft94\" (UID: \"fd6c8a79-0fec-4e70-a85c-6613f42ad6b9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94"
Sep 10 21:00:15.422149 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.422116    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:15.446210 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.446184    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:15.511052 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.511015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fd6c8a79-0fec-4e70-a85c-6613f42ad6b9-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-pft94\" (UID: \"fd6c8a79-0fec-4e70-a85c-6613f42ad6b9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94"
Sep 10 21:00:15.511227 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:00:15.511207    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 10 21:00:15.511306 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:00:15.511293    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd6c8a79-0fec-4e70-a85c-6613f42ad6b9-tls-certificates podName:fd6c8a79-0fec-4e70-a85c-6613f42ad6b9 nodeName:}" failed. No retries permitted until 2026-09-10 21:00:16.011270691 +0000 UTC m=+109.690769524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/fd6c8a79-0fec-4e70-a85c-6613f42ad6b9-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-pft94" (UID: "fd6c8a79-0fec-4e70-a85c-6613f42ad6b9") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 10 21:00:15.708978 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.708855    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84c98888f8-4jvjd"]
Sep 10 21:00:15.714701 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:00:15.714667    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod120bc529_f002_42fc_8ac6_653eacb78477.slice/crio-a1c2a432772a57c1429b0078447c9642d39f03e5d193ed8ed7e3198901360d46 WatchSource:0}: Error finding container a1c2a432772a57c1429b0078447c9642d39f03e5d193ed8ed7e3198901360d46: Status 404 returned error can't find the container with id a1c2a432772a57c1429b0078447c9642d39f03e5d193ed8ed7e3198901360d46
Sep 10 21:00:15.877537 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:15.877505    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:16.013420 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:16.013327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fd6c8a79-0fec-4e70-a85c-6613f42ad6b9-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-pft94\" (UID: \"fd6c8a79-0fec-4e70-a85c-6613f42ad6b9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94"
Sep 10 21:00:16.016144 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:16.016118    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/fd6c8a79-0fec-4e70-a85c-6613f42ad6b9-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-pft94\" (UID: \"fd6c8a79-0fec-4e70-a85c-6613f42ad6b9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94"
Sep 10 21:00:16.289012 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:16.288979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94"
Sep 10 21:00:16.439755 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:16.439718    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c98888f8-4jvjd" event={"ID":"120bc529-f002-42fc-8ac6-653eacb78477","Type":"ContainerStarted","Data":"a1c2a432772a57c1429b0078447c9642d39f03e5d193ed8ed7e3198901360d46"}
Sep 10 21:00:16.573077 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:16.572924    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94"]
Sep 10 21:00:16.578559 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:00:16.578527    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd6c8a79_0fec_4e70_a85c_6613f42ad6b9.slice/crio-b888986d50857a17e64766e162c84119a49c273475b1584bed82b0b306c7bd37 WatchSource:0}: Error finding container b888986d50857a17e64766e162c84119a49c273475b1584bed82b0b306c7bd37: Status 404 returned error can't find the container with id b888986d50857a17e64766e162c84119a49c273475b1584bed82b0b306c7bd37
Sep 10 21:00:17.445594 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:17.445556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94" event={"ID":"fd6c8a79-0fec-4e70-a85c-6613f42ad6b9","Type":"ContainerStarted","Data":"b888986d50857a17e64766e162c84119a49c273475b1584bed82b0b306c7bd37"}
Sep 10 21:00:17.526553 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:17.526518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 21:00:17.529115 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:17.529085    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b92e8363-eb18-4e60-8bce-9f5286beccfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vqzv4\" (UID: \"b92e8363-eb18-4e60-8bce-9f5286beccfa\") " pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 21:00:17.814442 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:17.814401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vqzv4"
Sep 10 21:00:18.415828 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:18.415793    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h2hwr"
Sep 10 21:00:20.800000 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:20.799966    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:20.857385 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:20.857354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:23.389733 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:23.389697    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:24.909223 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:24.909177    2580 patch_prober.go:28] interesting pod/image-registry-75555b558c-68ncn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 21:00:24.909683 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:24.909238    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-75555b558c-68ncn" podUID="9678ae86-ae81-4ade-8164-8f491017c3d2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:00:25.784253 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:25.784222    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:25.886826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:25.886738    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:26.094016 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.093986    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vqzv4"]
Sep 10 21:00:26.096788 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:00:26.096760    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb92e8363_eb18_4e60_8bce_9f5286beccfa.slice/crio-e65c1f498495839bf05738234c8bf32e93ef724bc18abdf9be12ca1b3c2efb1b WatchSource:0}: Error finding container e65c1f498495839bf05738234c8bf32e93ef724bc18abdf9be12ca1b3c2efb1b: Status 404 returned error can't find the container with id e65c1f498495839bf05738234c8bf32e93ef724bc18abdf9be12ca1b3c2efb1b
Sep 10 21:00:26.403646 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.403569    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 21:00:26.482892 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.481460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-k9dkd" event={"ID":"1d790ed2-e8f5-4eea-abd2-2af131a17bd8","Type":"ContainerStarted","Data":"57fafa496c058de2d839b134941146ba4c61a9f6faa05085a2a40d16b2d9a456"}
Sep 10 21:00:26.482892 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.481652    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-k9dkd"
Sep 10 21:00:26.486114 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.485696    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c98888f8-4jvjd" event={"ID":"120bc529-f002-42fc-8ac6-653eacb78477","Type":"ContainerStarted","Data":"5ab063546b395d2d277dcb1b4b09076968fb1ca79aa4308294c24b05f34fd7a1"}
Sep 10 21:00:26.488871 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.487630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vqzv4" event={"ID":"b92e8363-eb18-4e60-8bce-9f5286beccfa","Type":"ContainerStarted","Data":"e0599fc8a019509ab33faed244f40f0258a9777543a5802865a294b219a506e7"}
Sep 10 21:00:26.488871 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.487706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vqzv4" event={"ID":"b92e8363-eb18-4e60-8bce-9f5286beccfa","Type":"ContainerStarted","Data":"e65c1f498495839bf05738234c8bf32e93ef724bc18abdf9be12ca1b3c2efb1b"}
Sep 10 21:00:26.490475 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.490410    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94" event={"ID":"fd6c8a79-0fec-4e70-a85c-6613f42ad6b9","Type":"ContainerStarted","Data":"2abd9c7cb3a129bc660a8b5a4b085a393e6d626a06f68bc6bb8d4823186c34d1"}
Sep 10 21:00:26.490657 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.490635    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94"
Sep 10 21:00:26.496603 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.496579    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94"
Sep 10 21:00:26.500077 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.500045    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-k9dkd"
Sep 10 21:00:26.502913 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.502824    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-k9dkd" podStartSLOduration=2.247212561 podStartE2EDuration="19.502806057s" totalImagesPullingTime="17.255593496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:26.501740756 +0000 UTC m=+120.181239607" watchObservedRunningTime="2026-09-10 21:00:26.502806057 +0000 UTC m=+120.182304908"
Sep 10 21:00:26.547437 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.547387    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84c98888f8-4jvjd" podStartSLOduration=2.440222782 podStartE2EDuration="12.54736661s" totalImagesPullingTime="10.107143828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:26.546474667 +0000 UTC m=+120.225973511" watchObservedRunningTime="2026-09-10 21:00:26.54736661 +0000 UTC m=+120.226865461"
Sep 10 21:00:26.630309 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.630273    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:26.726801 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:26.726722    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:27.404325 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:27.404289    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:27.481958 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:27.481919    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:27.497860 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:27.497828    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vqzv4" event={"ID":"b92e8363-eb18-4e60-8bce-9f5286beccfa","Type":"ContainerStarted","Data":"fa379a44fb4ea52a836bbec7bb59406ec497f032cd8588287661028eb4031593"}
Sep 10 21:00:27.539109 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:27.539070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:27.595893 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:27.595863    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:29.506972 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:29.506930    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vqzv4" event={"ID":"b92e8363-eb18-4e60-8bce-9f5286beccfa","Type":"ContainerStarted","Data":"3e7a1288aff71e7328db9a76d4efa728d63dc5edfdee73a6405db371a7a55bb4"}
Sep 10 21:00:29.526413 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:29.526358    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vqzv4" podStartSLOduration=41.88269688 podStartE2EDuration="44.526340878s" totalImagesPullingTime="2.643643998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:59:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:29.525864191 +0000 UTC m=+123.205363041" watchObservedRunningTime="2026-09-10 21:00:29.526340878 +0000 UTC m=+123.205839728"
Sep 10 21:00:29.526912 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:29.526877    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pft94" podStartSLOduration=5.28890083 podStartE2EDuration="14.526866773s" totalImagesPullingTime="9.237965943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:26.564313515 +0000 UTC m=+120.243812366" watchObservedRunningTime="2026-09-10 21:00:29.526866773 +0000 UTC m=+123.206365623"
Sep 10 21:00:29.713496 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:29.713462    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:30.236349 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:30.236315    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75555b558c-68ncn"]
Sep 10 21:00:30.348229 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:30.348195    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:30.414185 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:30.414157    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:31.555780 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.555752    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:31.791789 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.791755    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:31.855735 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.855635    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"]
Sep 10 21:00:31.884875 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.884839    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:31.889260 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.889240    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"]
Sep 10 21:00:31.889407 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.889381    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:31.889534 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.889419    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xs425"]
Sep 10 21:00:31.892307 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.892289    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 21:00:31.892631 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.892296    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 21:00:31.892631 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.892353    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 21:00:31.894202 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.893413    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 21:00:31.894202 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.893666    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 21:00:31.894202 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.893873    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-875qw\""
Sep 10 21:00:31.895512 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.894698    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 21:00:31.915850 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.915831    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:31.920488 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.919288    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 21:00:31.920859 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.920842    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rpll2\""
Sep 10 21:00:31.921349 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.921334    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 21:00:31.922413 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.922265    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 21:00:31.939172 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.938858    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/45829093-179f-41e5-b924-0046354ad7b0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:31.939172 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.938923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45829093-179f-41e5-b924-0046354ad7b0-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:31.939172 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.938972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/45829093-179f-41e5-b924-0046354ad7b0-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:31.939172 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.938999    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45829093-179f-41e5-b924-0046354ad7b0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:31.939172 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.939024    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st62g\" (UniqueName: \"kubernetes.io/projected/45829093-179f-41e5-b924-0046354ad7b0-kube-api-access-st62g\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:31.939172 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:31.939078    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45829093-179f-41e5-b924-0046354ad7b0-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.015698 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.015666    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:32.039731 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.039701    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-wtmp\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.039845 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.039743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtbrx\" (UniqueName: \"kubernetes.io/projected/7cdc81b7-3624-4c4d-a96a-de94386c5a93-kube-api-access-mtbrx\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.039845 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.039779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/45829093-179f-41e5-b924-0046354ad7b0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.039955 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.039870    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cdc81b7-3624-4c4d-a96a-de94386c5a93-metrics-client-ca\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.039955 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.039909    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-tls\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.039955 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.039949    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cdc81b7-3624-4c4d-a96a-de94386c5a93-sys\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.040121 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.039976    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.040121 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.040008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-textfile\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.040121 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.040038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45829093-179f-41e5-b924-0046354ad7b0-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.040121 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.040102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/45829093-179f-41e5-b924-0046354ad7b0-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.040323 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.040132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45829093-179f-41e5-b924-0046354ad7b0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.040323 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.040159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st62g\" (UniqueName: \"kubernetes.io/projected/45829093-179f-41e5-b924-0046354ad7b0-kube-api-access-st62g\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.040323 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.040284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-accelerators-collector-config\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.040482 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.040325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7cdc81b7-3624-4c4d-a96a-de94386c5a93-root\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.040482 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.040369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45829093-179f-41e5-b924-0046354ad7b0-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.040582 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.040487    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/45829093-179f-41e5-b924-0046354ad7b0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.040732 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.040712    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45829093-179f-41e5-b924-0046354ad7b0-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.040879 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.040726    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/45829093-179f-41e5-b924-0046354ad7b0-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.043310 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.043289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45829093-179f-41e5-b924-0046354ad7b0-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.043464 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.043421    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45829093-179f-41e5-b924-0046354ad7b0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.050127 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.050103    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st62g\" (UniqueName: \"kubernetes.io/projected/45829093-179f-41e5-b924-0046354ad7b0-kube-api-access-st62g\") pod \"kube-state-metrics-59fbdd77d-wj7qk\" (UID: \"45829093-179f-41e5-b924-0046354ad7b0\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.117508 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.117446    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:32.140951 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.140922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-accelerators-collector-config\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.141075 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.140961    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7cdc81b7-3624-4c4d-a96a-de94386c5a93-root\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.141075 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.140990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-wtmp\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.141075 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.141011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtbrx\" (UniqueName: \"kubernetes.io/projected/7cdc81b7-3624-4c4d-a96a-de94386c5a93-kube-api-access-mtbrx\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.141075 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.141039    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7cdc81b7-3624-4c4d-a96a-de94386c5a93-root\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.141290 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.141077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cdc81b7-3624-4c4d-a96a-de94386c5a93-metrics-client-ca\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.141290 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.141106    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-tls\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.141290 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.141133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cdc81b7-3624-4c4d-a96a-de94386c5a93-sys\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.141290 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.141155    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.141290 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.141181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-textfile\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.141290 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.141190    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-wtmp\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.141290 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.141277    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cdc81b7-3624-4c4d-a96a-de94386c5a93-sys\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.141631 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:00:32.141297    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 21:00:32.141631 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:00:32.141354    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-tls podName:7cdc81b7-3624-4c4d-a96a-de94386c5a93 nodeName:}" failed. No retries permitted until 2026-09-10 21:00:32.641334551 +0000 UTC m=+126.320833379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-tls") pod "node-exporter-xs425" (UID: "7cdc81b7-3624-4c4d-a96a-de94386c5a93") : secret "node-exporter-tls" not found
Sep 10 21:00:32.141929 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.141899    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-textfile\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.142316 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.142268    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cdc81b7-3624-4c4d-a96a-de94386c5a93-metrics-client-ca\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.142806 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.142781    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-accelerators-collector-config\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.143761 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.143720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.148369 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.148343    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84c98888f8-4jvjd"]
Sep 10 21:00:32.151619 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.151599    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtbrx\" (UniqueName: \"kubernetes.io/projected/7cdc81b7-3624-4c4d-a96a-de94386c5a93-kube-api-access-mtbrx\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.205073 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.204728    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"
Sep 10 21:00:32.248266 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.243952    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:32.345680 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.345645    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:32.444480 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.444415    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:32.537008 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.536973    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk"]
Sep 10 21:00:32.539814 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:00:32.539786    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45829093_179f_41e5_b924_0046354ad7b0.slice/crio-da0cbbf51685983525cb03b946ddf50e3e38973255d4d38c2f5127274ffbd550 WatchSource:0}: Error finding container da0cbbf51685983525cb03b946ddf50e3e38973255d4d38c2f5127274ffbd550: Status 404 returned error can't find the container with id da0cbbf51685983525cb03b946ddf50e3e38973255d4d38c2f5127274ffbd550
Sep 10 21:00:32.644613 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.644579    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-tls\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:32.645008 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:00:32.644766    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 21:00:32.645008 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:00:32.644856    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-tls podName:7cdc81b7-3624-4c4d-a96a-de94386c5a93 nodeName:}" failed. No retries permitted until 2026-09-10 21:00:33.644822411 +0000 UTC m=+127.324321242 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-tls") pod "node-exporter-xs425" (UID: "7cdc81b7-3624-4c4d-a96a-de94386c5a93") : secret "node-exporter-tls" not found
Sep 10 21:00:32.771857 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.771782    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:32.877616 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.877582    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:32.984114 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:32.984077    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:33.032780 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.032718    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 21:00:33.058300 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.058274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.061864 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.061708    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 21:00:33.061864 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.061848    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 21:00:33.062393 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.062366    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 21:00:33.062681 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.062507    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-gd8bd\""
Sep 10 21:00:33.063317 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.063292    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 21:00:33.063415 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.063380    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 21:00:33.064710 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.064693    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 21:00:33.065936 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.065854    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 21:00:33.066535 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.066515    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 21:00:33.072328 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.072309    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 21:00:33.074082 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.074045    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 21:00:33.148031 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.147998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.148170 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.148047    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/310a9670-2b9b-4762-bcf4-bcc91747d22f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.148170 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.148146    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.148293 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.148193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-web-config\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.148293 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.148229    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.148293 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.148267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.148452 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.148296    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/310a9670-2b9b-4762-bcf4-bcc91747d22f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.148452 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.148323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/310a9670-2b9b-4762-bcf4-bcc91747d22f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.148452 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.148356    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mqnd\" (UniqueName: \"kubernetes.io/projected/310a9670-2b9b-4762-bcf4-bcc91747d22f-kube-api-access-9mqnd\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.148452 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.148408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-config-volume\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.148452 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.148447    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.148664 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.148477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/310a9670-2b9b-4762-bcf4-bcc91747d22f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.148664 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.148505    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/310a9670-2b9b-4762-bcf4-bcc91747d22f-config-out\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.180838 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.180746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:33.249818 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.249782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/310a9670-2b9b-4762-bcf4-bcc91747d22f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.249959 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.249831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/310a9670-2b9b-4762-bcf4-bcc91747d22f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.249959 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.249883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mqnd\" (UniqueName: \"kubernetes.io/projected/310a9670-2b9b-4762-bcf4-bcc91747d22f-kube-api-access-9mqnd\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.249959 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.249913    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-config-volume\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.249959 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.249946    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.250199 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.249978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/310a9670-2b9b-4762-bcf4-bcc91747d22f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.250199 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.250005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/310a9670-2b9b-4762-bcf4-bcc91747d22f-config-out\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.250199 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.250040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.250199 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.250117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/310a9670-2b9b-4762-bcf4-bcc91747d22f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.250199 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.250163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.250199 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.250195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-web-config\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.250481 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.250231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.250481 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.250276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.253068 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.251422    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/310a9670-2b9b-4762-bcf4-bcc91747d22f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.253068 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.252415    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/310a9670-2b9b-4762-bcf4-bcc91747d22f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.253068 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.252499    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/310a9670-2b9b-4762-bcf4-bcc91747d22f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.257669 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.257627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.258446 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.257821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-config-volume\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.258446 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.257986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/310a9670-2b9b-4762-bcf4-bcc91747d22f-config-out\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.258446 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.258254    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.258446 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.258398    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.259759 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.259711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-web-config\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.259939 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.259915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.264431 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.263783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/310a9670-2b9b-4762-bcf4-bcc91747d22f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.264431 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.264176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mqnd\" (UniqueName: \"kubernetes.io/projected/310a9670-2b9b-4762-bcf4-bcc91747d22f-kube-api-access-9mqnd\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.265025 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.265000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/310a9670-2b9b-4762-bcf4-bcc91747d22f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"310a9670-2b9b-4762-bcf4-bcc91747d22f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.279176 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.279148    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:33.370729 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.370686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 21:00:33.532535 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.532473    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk" event={"ID":"45829093-179f-41e5-b924-0046354ad7b0","Type":"ContainerStarted","Data":"da0cbbf51685983525cb03b946ddf50e3e38973255d4d38c2f5127274ffbd550"}
Sep 10 21:00:33.654367 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.654290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-tls\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:33.656960 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.656932    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7cdc81b7-3624-4c4d-a96a-de94386c5a93-node-exporter-tls\") pod \"node-exporter-xs425\" (UID: \"7cdc81b7-3624-4c4d-a96a-de94386c5a93\") " pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:33.704902 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.704866    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 21:00:33.728691 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.728666    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xs425"
Sep 10 21:00:33.934354 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.934268    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5489f8965d-msdmg"]
Sep 10 21:00:33.953149 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.952405    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:33.953149 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.952635    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5489f8965d-msdmg"]
Sep 10 21:00:33.957154 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.956078    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 21:00:33.957154 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.956744    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 21:00:33.957154 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.957009    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9eh1n1734joqi\""
Sep 10 21:00:33.957373 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.957265    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 21:00:33.957458 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.957433    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 21:00:33.957607 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.957590    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 21:00:33.957842 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:33.957821    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-mlkk2\""
Sep 10 21:00:34.040140 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:00:34.040111    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod310a9670_2b9b_4762_bcf4_bcc91747d22f.slice/crio-ffdd62217ef1e1fc9253a409bdba634c810bc6c1b4cf03a1f3ba36ed091ae06d WatchSource:0}: Error finding container ffdd62217ef1e1fc9253a409bdba634c810bc6c1b4cf03a1f3ba36ed091ae06d: Status 404 returned error can't find the container with id ffdd62217ef1e1fc9253a409bdba634c810bc6c1b4cf03a1f3ba36ed091ae06d
Sep 10 21:00:34.047642 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:00:34.047620    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cdc81b7_3624_4c4d_a96a_de94386c5a93.slice/crio-2fe97fb83c884d4f533f34e0cea4c2612f837d2fc3108f83f257b776228a382d WatchSource:0}: Error finding container 2fe97fb83c884d4f533f34e0cea4c2612f837d2fc3108f83f257b776228a382d: Status 404 returned error can't find the container with id 2fe97fb83c884d4f533f34e0cea4c2612f837d2fc3108f83f257b776228a382d
Sep 10 21:00:34.058044 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.058002    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51081d47-258b-4956-8961-c90f29034350-metrics-client-ca\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.058161 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.058049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.058161 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.058094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-grpc-tls\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.058161 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.058132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-tls\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.058310 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.058232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.058310 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.058281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9dtc\" (UniqueName: \"kubernetes.io/projected/51081d47-258b-4956-8961-c90f29034350-kube-api-access-z9dtc\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.058409 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.058350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.058458 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.058443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.159392 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.159357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.159520 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.159414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9dtc\" (UniqueName: \"kubernetes.io/projected/51081d47-258b-4956-8961-c90f29034350-kube-api-access-z9dtc\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.159520 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.159448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.159520 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.159493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.159691 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.159527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51081d47-258b-4956-8961-c90f29034350-metrics-client-ca\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.159691 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.159565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.159691 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.159589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-grpc-tls\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.159691 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.159628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-tls\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.160556 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.160527    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51081d47-258b-4956-8961-c90f29034350-metrics-client-ca\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.162525 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.162504    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.162707 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.162663    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.162813 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.162709    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-tls\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.163165 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.163143    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.163493 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.163467    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.164546 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.164526    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/51081d47-258b-4956-8961-c90f29034350-secret-grpc-tls\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.170069 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.170032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9dtc\" (UniqueName: \"kubernetes.io/projected/51081d47-258b-4956-8961-c90f29034350-kube-api-access-z9dtc\") pod \"thanos-querier-5489f8965d-msdmg\" (UID: \"51081d47-258b-4956-8961-c90f29034350\") " pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.266617 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.266557    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:34.542302 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.542265    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xs425" event={"ID":"7cdc81b7-3624-4c4d-a96a-de94386c5a93","Type":"ContainerStarted","Data":"2fe97fb83c884d4f533f34e0cea4c2612f837d2fc3108f83f257b776228a382d"}
Sep 10 21:00:34.544323 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.544000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"310a9670-2b9b-4762-bcf4-bcc91747d22f","Type":"ContainerStarted","Data":"ffdd62217ef1e1fc9253a409bdba634c810bc6c1b4cf03a1f3ba36ed091ae06d"}
Sep 10 21:00:34.805655 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.805444    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5489f8965d-msdmg"]
Sep 10 21:00:34.811129 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:00:34.810699    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51081d47_258b_4956_8961_c90f29034350.slice/crio-0ba39a20f0b4dd952270ba294107859818ee6b64d3f89657e0c2488dd27f1773 WatchSource:0}: Error finding container 0ba39a20f0b4dd952270ba294107859818ee6b64d3f89657e0c2488dd27f1773: Status 404 returned error can't find the container with id 0ba39a20f0b4dd952270ba294107859818ee6b64d3f89657e0c2488dd27f1773
Sep 10 21:00:34.871990 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.871931    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:34.985968 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:34.985940    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:35.096425 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:35.096394    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:35.188006 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:35.187967    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:35.422719 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:35.422655    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:35.509295 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:35.509265    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:35.550658 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:35.550626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk" event={"ID":"45829093-179f-41e5-b924-0046354ad7b0","Type":"ContainerStarted","Data":"e727b18b59de9a01eb8094150b3fde697abafc51766ef354733c0c2f10515cdb"}
Sep 10 21:00:35.550803 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:35.550690    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk" event={"ID":"45829093-179f-41e5-b924-0046354ad7b0","Type":"ContainerStarted","Data":"412d7c642be6bb1a7e760727e4bf814d3ed4094a569fa431091e8c114e3a9fb1"}
Sep 10 21:00:35.550803 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:35.550720    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk" event={"ID":"45829093-179f-41e5-b924-0046354ad7b0","Type":"ContainerStarted","Data":"8ce1943eb98bba6c0d8d41a64e81b7c1dfedaeecc6a69cf9687585722f0e4e0f"}
Sep 10 21:00:35.552131 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:35.552096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg" event={"ID":"51081d47-258b-4956-8961-c90f29034350","Type":"ContainerStarted","Data":"0ba39a20f0b4dd952270ba294107859818ee6b64d3f89657e0c2488dd27f1773"}
Sep 10 21:00:35.588372 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:35.588327    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-wj7qk" podStartSLOduration=2.650110911 podStartE2EDuration="4.588311679s" totalImagesPullingTime="1.938200768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:35.588221659 +0000 UTC m=+129.267720510" watchObservedRunningTime="2026-09-10 21:00:35.588311679 +0000 UTC m=+129.267810528"
Sep 10 21:00:35.595006 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:35.594982    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:36.556833 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.556802    2580 generic.go:356] "Generic (PLEG): container finished" podID="7cdc81b7-3624-4c4d-a96a-de94386c5a93" containerID="99ea23fc3cfa83afc94520bc17ab7abb24f7490b115c28bb653f13c0807f20f3" exitCode=0
Sep 10 21:00:36.557229 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.556889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xs425" event={"ID":"7cdc81b7-3624-4c4d-a96a-de94386c5a93","Type":"ContainerDied","Data":"99ea23fc3cfa83afc94520bc17ab7abb24f7490b115c28bb653f13c0807f20f3"}
Sep 10 21:00:36.558518 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.558499    2580 generic.go:356] "Generic (PLEG): container finished" podID="310a9670-2b9b-4762-bcf4-bcc91747d22f" containerID="87c7e3fcdaf331301c7687ce15a094cdc23d774134a61038b8ac8ae1d548cbb9" exitCode=0
Sep 10 21:00:36.558640 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.558583    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"310a9670-2b9b-4762-bcf4-bcc91747d22f","Type":"ContainerDied","Data":"87c7e3fcdaf331301c7687ce15a094cdc23d774134a61038b8ac8ae1d548cbb9"}
Sep 10 21:00:36.580338 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.580314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 21:00:36.583320 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.583301    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bb0820e-193e-416f-a950-5eedb01fc196-metrics-certs\") pod \"network-metrics-daemon-dkc7w\" (UID: \"1bb0820e-193e-416f-a950-5eedb01fc196\") " pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 21:00:36.668901 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.668881    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5khf6\""
Sep 10 21:00:36.675573 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.675558    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkc7w"
Sep 10 21:00:36.719508 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.719485    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:36.762285 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.760145    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-569bf6b96c-lqqc8"]
Sep 10 21:00:36.782771 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.782746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.790469 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.790417    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-569bf6b96c-lqqc8"]
Sep 10 21:00:36.799670 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.799639    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 21:00:36.841316 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.841246    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:36.883113 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.883084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-console-config\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.883235 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.883146    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-trusted-ca-bundle\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.883235 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.883217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-oauth-serving-cert\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.883315 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.883270    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4782740c-925f-4468-b14c-65a639fcd905-console-serving-cert\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.883315 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.883297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-service-ca\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.883388 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.883319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4782740c-925f-4468-b14c-65a639fcd905-console-oauth-config\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.883388 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.883336    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqz6x\" (UniqueName: \"kubernetes.io/projected/4782740c-925f-4468-b14c-65a639fcd905-kube-api-access-pqz6x\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.984646 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.984619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-service-ca\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.984799 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.984655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4782740c-925f-4468-b14c-65a639fcd905-console-oauth-config\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.984799 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.984677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqz6x\" (UniqueName: \"kubernetes.io/projected/4782740c-925f-4468-b14c-65a639fcd905-kube-api-access-pqz6x\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.984799 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.984752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-console-config\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.984799 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.984778    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-trusted-ca-bundle\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.985047 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.984829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-oauth-serving-cert\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.985047 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.984859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4782740c-925f-4468-b14c-65a639fcd905-console-serving-cert\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.985473 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.985443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-service-ca\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.985683 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.985662    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-oauth-serving-cert\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.985768 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.985718    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-console-config\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.986209 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.986186    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-trusted-ca-bundle\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.987531 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.987508    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4782740c-925f-4468-b14c-65a639fcd905-console-oauth-config\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.987889 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.987873    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4782740c-925f-4468-b14c-65a639fcd905-console-serving-cert\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:36.994573 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:36.994550    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dkc7w"]
Sep 10 21:00:36.997982 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:00:36.997957    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bb0820e_193e_416f_a950_5eedb01fc196.slice/crio-364cb3716809d0080f9ea14dd84698a75c86fdd6dae8f652e9aa933a34da7007 WatchSource:0}: Error finding container 364cb3716809d0080f9ea14dd84698a75c86fdd6dae8f652e9aa933a34da7007: Status 404 returned error can't find the container with id 364cb3716809d0080f9ea14dd84698a75c86fdd6dae8f652e9aa933a34da7007
Sep 10 21:00:37.000593 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:37.000399    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqz6x\" (UniqueName: \"kubernetes.io/projected/4782740c-925f-4468-b14c-65a639fcd905-kube-api-access-pqz6x\") pod \"console-569bf6b96c-lqqc8\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") " pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:37.000593 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:37.000538    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:37.080666 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:37.080612    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:37.096142 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:37.096121    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:37.450668 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:37.450642    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-569bf6b96c-lqqc8"]
Sep 10 21:00:37.565977 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:37.565939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xs425" event={"ID":"7cdc81b7-3624-4c4d-a96a-de94386c5a93","Type":"ContainerStarted","Data":"d9dd52515a71fc351c29643024ef385a6bd77a72aecf7d0a62e1fd752310e6f4"}
Sep 10 21:00:37.566412 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:37.566006    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xs425" event={"ID":"7cdc81b7-3624-4c4d-a96a-de94386c5a93","Type":"ContainerStarted","Data":"8e9e0ffe7a1fabed229456fde4d06ea00b43dee5716ebb842b426c808863e218"}
Sep 10 21:00:37.567940 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:37.567914    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dkc7w" event={"ID":"1bb0820e-193e-416f-a950-5eedb01fc196","Type":"ContainerStarted","Data":"364cb3716809d0080f9ea14dd84698a75c86fdd6dae8f652e9aa933a34da7007"}
Sep 10 21:00:37.594643 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:37.594509    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xs425" podStartSLOduration=4.492287706 podStartE2EDuration="6.594491935s" totalImagesPullingTime="2.026459774s" totalInitContainerRuntime="75.744455ms" isStatefulPod=true podCreationTimestamp="2026-09-10 21:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:37.593151473 +0000 UTC m=+131.272650324" watchObservedRunningTime="2026-09-10 21:00:37.594491935 +0000 UTC m=+131.273990788"
Sep 10 21:00:37.710435 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:37.710353    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:37.809584 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:37.809548    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:37.997756 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:00:37.997687    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4782740c_925f_4468_b14c_65a639fcd905.slice/crio-d331541a16fa6bc5277be17d6bae346878a40c1f64bdb0d8bd3c177a9942ddde WatchSource:0}: Error finding container d331541a16fa6bc5277be17d6bae346878a40c1f64bdb0d8bd3c177a9942ddde: Status 404 returned error can't find the container with id d331541a16fa6bc5277be17d6bae346878a40c1f64bdb0d8bd3c177a9942ddde
Sep 10 21:00:38.313280 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:38.313234    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:38.572774 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:38.572711    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569bf6b96c-lqqc8" event={"ID":"4782740c-925f-4468-b14c-65a639fcd905","Type":"ContainerStarted","Data":"d331541a16fa6bc5277be17d6bae346878a40c1f64bdb0d8bd3c177a9942ddde"}
Sep 10 21:00:39.578804 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.578774    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569bf6b96c-lqqc8" event={"ID":"4782740c-925f-4468-b14c-65a639fcd905","Type":"ContainerStarted","Data":"f947f3efcc48ab4a2c2f88b51892b648d25a89627f3161ae6bd559428d4e59bf"}
Sep 10 21:00:39.582637 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.582606    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"310a9670-2b9b-4762-bcf4-bcc91747d22f","Type":"ContainerStarted","Data":"9d2bbe0c80706fd2092c9dd1f4d13f36132796df5ff7164b0c057c077acd0fea"}
Sep 10 21:00:39.582742 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.582685    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"310a9670-2b9b-4762-bcf4-bcc91747d22f","Type":"ContainerStarted","Data":"e07007bb23f3154ed82aa9be0a9d0174785026c2c9e1c74e7c6153e06e59f860"}
Sep 10 21:00:39.582742 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.582721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"310a9670-2b9b-4762-bcf4-bcc91747d22f","Type":"ContainerStarted","Data":"adecb6bdc7f35bd5b185fa307a74b048a2c4c5fdec7106f4d7800f0202ba556f"}
Sep 10 21:00:39.582827 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.582745    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"310a9670-2b9b-4762-bcf4-bcc91747d22f","Type":"ContainerStarted","Data":"99c169733f9003a3a903f49f49b1173de297ecd7babb187a1eec12df912d9fa8"}
Sep 10 21:00:39.582827 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.582776    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"310a9670-2b9b-4762-bcf4-bcc91747d22f","Type":"ContainerStarted","Data":"d673074ea4083c3979329ac1b74cad5d53d857d65a4378c6ada4d438d799b1e2"}
Sep 10 21:00:39.584800 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.584778    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg" event={"ID":"51081d47-258b-4956-8961-c90f29034350","Type":"ContainerStarted","Data":"76212252049def2fdbe8c183a9f8678bd41704f1416ee63afb99891591869c5c"}
Sep 10 21:00:39.584903 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.584849    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg" event={"ID":"51081d47-258b-4956-8961-c90f29034350","Type":"ContainerStarted","Data":"ee0d3300acc53ec4a2bda331088d19dda66429286b7cfa5728df05f602e66424"}
Sep 10 21:00:39.584903 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.584893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg" event={"ID":"51081d47-258b-4956-8961-c90f29034350","Type":"ContainerStarted","Data":"324deac7de446b450066f6f8567bd85daefdc612c590c06721ef144481bfefdb"}
Sep 10 21:00:39.586582 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.586555    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dkc7w" event={"ID":"1bb0820e-193e-416f-a950-5eedb01fc196","Type":"ContainerStarted","Data":"7225b2b30d0e99c390f7aa2636915b417931fc0780fa34702e88f281276ffa11"}
Sep 10 21:00:39.586672 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.586605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dkc7w" event={"ID":"1bb0820e-193e-416f-a950-5eedb01fc196","Type":"ContainerStarted","Data":"125564d42c6a2f59ac0dddeb6ede5e91de0e81bc7c54005a1ebb4d8de8ac8b3a"}
Sep 10 21:00:39.607922 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.607875    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-569bf6b96c-lqqc8" podStartSLOduration=3.607861282 podStartE2EDuration="3.607861282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:39.607309892 +0000 UTC m=+133.286808743" watchObservedRunningTime="2026-09-10 21:00:39.607861282 +0000 UTC m=+133.287360132"
Sep 10 21:00:39.629250 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.629213    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dkc7w" podStartSLOduration=130.36009808 podStartE2EDuration="2m12.629199663s" totalImagesPullingTime="2.269101583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:39.628350204 +0000 UTC m=+133.307849055" watchObservedRunningTime="2026-09-10 21:00:39.629199663 +0000 UTC m=+133.308698514"
Sep 10 21:00:39.758109 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.758032    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:39.830559 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.830533    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:39.925948 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:39.925922    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:40.704185 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:40.704152    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:40.803515 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:40.803487    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:41.598365 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:41.598338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"310a9670-2b9b-4762-bcf4-bcc91747d22f","Type":"ContainerStarted","Data":"958882eab4f995a861fea9d3b845f5516660e3481dc915f5143109759cf4fd8e"}
Sep 10 21:00:41.603563 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:41.603534    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg" event={"ID":"51081d47-258b-4956-8961-c90f29034350","Type":"ContainerStarted","Data":"c68caff141d19d52538b4bc01984bc52ceec3528b3b6d3fdf2a255934e7dba00"}
Sep 10 21:00:41.603683 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:41.603614    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:41.603683 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:41.603651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg" event={"ID":"51081d47-258b-4956-8961-c90f29034350","Type":"ContainerStarted","Data":"80589cd0014eac4573fa01bc07394290326852d3c53bed21246e157881c7393b"}
Sep 10 21:00:41.603755 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:41.603700    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg" event={"ID":"51081d47-258b-4956-8961-c90f29034350","Type":"ContainerStarted","Data":"2a7f3ab2bdcf59506c0bfba266b3bdde0a34737a8e48f60d39cde689fd877228"}
Sep 10 21:00:41.645430 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:41.645387    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.902401345 podStartE2EDuration="9.645372865s" totalImagesPullingTime="5.717613497s" totalInitContainerRuntime="25.358023ms" isStatefulPod=false podCreationTimestamp="2026-09-10 21:00:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:41.645206659 +0000 UTC m=+135.324705508" watchObservedRunningTime="2026-09-10 21:00:41.645372865 +0000 UTC m=+135.324871715"
Sep 10 21:00:41.680598 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:41.680546    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg" podStartSLOduration=3.049610598 podStartE2EDuration="8.680524717s" totalImagesPullingTime="5.630914119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:00:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:41.678583186 +0000 UTC m=+135.358082036" watchObservedRunningTime="2026-09-10 21:00:41.680524717 +0000 UTC m=+135.360023568"
Sep 10 21:00:42.237808 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:42.237753    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:42.339632 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:42.339597    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:45.272639 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:45.272597    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:45.374195 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:45.374160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:45.476258 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:45.476224    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:46.350665 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:46.350630    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:46.465251 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:46.465224    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:47.096288 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:47.096254    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:47.096455 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:47.096312    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:47.101030 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:47.101012    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:47.614703 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:47.614670    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5489f8965d-msdmg"
Sep 10 21:00:47.631416 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:47.631391    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:00:47.869454 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:47.869377    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:47.972030 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:47.971993    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:49.801228 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:49.801192    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:49.902730 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:49.902695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:00:55.258110 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.258038    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-75555b558c-68ncn" podUID="9678ae86-ae81-4ade-8164-8f491017c3d2" containerName="registry" containerID="cri-o://650b84013224e5d7787aa4498e1936218794050b2e70ea646b06881c8279474e" gracePeriod=30
Sep 10 21:00:55.532740 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.532673    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 21:00:55.644304 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.644232    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9678ae86-ae81-4ade-8164-8f491017c3d2-ca-trust-extracted\") pod \"9678ae86-ae81-4ade-8164-8f491017c3d2\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") "
Sep 10 21:00:55.644304 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.644285    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9678ae86-ae81-4ade-8164-8f491017c3d2-installation-pull-secrets\") pod \"9678ae86-ae81-4ade-8164-8f491017c3d2\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") "
Sep 10 21:00:55.644545 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.644339    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-bound-sa-token\") pod \"9678ae86-ae81-4ade-8164-8f491017c3d2\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") "
Sep 10 21:00:55.644545 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.644361    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-certificates\") pod \"9678ae86-ae81-4ade-8164-8f491017c3d2\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") "
Sep 10 21:00:55.644545 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.644384    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls\") pod \"9678ae86-ae81-4ade-8164-8f491017c3d2\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") "
Sep 10 21:00:55.644545 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.644407    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9678ae86-ae81-4ade-8164-8f491017c3d2-trusted-ca\") pod \"9678ae86-ae81-4ade-8164-8f491017c3d2\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") "
Sep 10 21:00:55.644545 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.644432    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjvjt\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-kube-api-access-gjvjt\") pod \"9678ae86-ae81-4ade-8164-8f491017c3d2\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") "
Sep 10 21:00:55.644545 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.644488    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9678ae86-ae81-4ade-8164-8f491017c3d2-image-registry-private-configuration\") pod \"9678ae86-ae81-4ade-8164-8f491017c3d2\" (UID: \"9678ae86-ae81-4ade-8164-8f491017c3d2\") "
Sep 10 21:00:55.645017 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.644953    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9678ae86-ae81-4ade-8164-8f491017c3d2" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:00:55.645017 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.644975    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9678ae86-ae81-4ade-8164-8f491017c3d2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9678ae86-ae81-4ade-8164-8f491017c3d2" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:00:55.647051 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.647023    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9678ae86-ae81-4ade-8164-8f491017c3d2" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:00:55.647304 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.647184    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-kube-api-access-gjvjt" (OuterVolumeSpecName: "kube-api-access-gjvjt") pod "9678ae86-ae81-4ade-8164-8f491017c3d2" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2"). InnerVolumeSpecName "kube-api-access-gjvjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:00:55.647380 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.647296    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9678ae86-ae81-4ade-8164-8f491017c3d2" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:00:55.647380 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.647315    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9678ae86-ae81-4ade-8164-8f491017c3d2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9678ae86-ae81-4ade-8164-8f491017c3d2" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:00:55.647527 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.647506    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9678ae86-ae81-4ade-8164-8f491017c3d2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9678ae86-ae81-4ade-8164-8f491017c3d2" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:00:55.652982 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.652956    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9678ae86-ae81-4ade-8164-8f491017c3d2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9678ae86-ae81-4ade-8164-8f491017c3d2" (UID: "9678ae86-ae81-4ade-8164-8f491017c3d2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:00:55.659304 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.659281    2580 generic.go:356] "Generic (PLEG): container finished" podID="9678ae86-ae81-4ade-8164-8f491017c3d2" containerID="650b84013224e5d7787aa4498e1936218794050b2e70ea646b06881c8279474e" exitCode=0
Sep 10 21:00:55.659394 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.659316    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75555b558c-68ncn" event={"ID":"9678ae86-ae81-4ade-8164-8f491017c3d2","Type":"ContainerDied","Data":"650b84013224e5d7787aa4498e1936218794050b2e70ea646b06881c8279474e"}
Sep 10 21:00:55.659394 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.659324    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75555b558c-68ncn"
Sep 10 21:00:55.659394 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.659378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75555b558c-68ncn" event={"ID":"9678ae86-ae81-4ade-8164-8f491017c3d2","Type":"ContainerDied","Data":"bbc59eafac1e531917565f1ad91537ab83db31815a4487c49dab1334ef35346a"}
Sep 10 21:00:55.659555 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.659398    2580 scope.go:117] "RemoveContainer" containerID="650b84013224e5d7787aa4498e1936218794050b2e70ea646b06881c8279474e"
Sep 10 21:00:55.667679 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.667662    2580 scope.go:117] "RemoveContainer" containerID="650b84013224e5d7787aa4498e1936218794050b2e70ea646b06881c8279474e"
Sep 10 21:00:55.667979 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:00:55.667960    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"650b84013224e5d7787aa4498e1936218794050b2e70ea646b06881c8279474e\": container with ID starting with 650b84013224e5d7787aa4498e1936218794050b2e70ea646b06881c8279474e not found: ID does not exist" containerID="650b84013224e5d7787aa4498e1936218794050b2e70ea646b06881c8279474e"
Sep 10 21:00:55.668040 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.667985    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"650b84013224e5d7787aa4498e1936218794050b2e70ea646b06881c8279474e"} err="failed to get container status \"650b84013224e5d7787aa4498e1936218794050b2e70ea646b06881c8279474e\": rpc error: code = NotFound desc = could not find container \"650b84013224e5d7787aa4498e1936218794050b2e70ea646b06881c8279474e\": container with ID starting with 650b84013224e5d7787aa4498e1936218794050b2e70ea646b06881c8279474e not found: ID does not exist"
Sep 10 21:00:55.684870 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.684841    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75555b558c-68ncn"]
Sep 10 21:00:55.687472 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.687453    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75555b558c-68ncn"]
Sep 10 21:00:55.745464 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.745444    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9678ae86-ae81-4ade-8164-8f491017c3d2-image-registry-private-configuration\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:55.745464 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.745463    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9678ae86-ae81-4ade-8164-8f491017c3d2-ca-trust-extracted\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:55.745589 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.745475    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9678ae86-ae81-4ade-8164-8f491017c3d2-installation-pull-secrets\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:55.745589 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.745485    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-bound-sa-token\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:55.745589 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.745493    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-certificates\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:55.745589 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.745501    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-registry-tls\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:55.745589 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.745509    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9678ae86-ae81-4ade-8164-8f491017c3d2-trusted-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:55.745589 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:55.745516    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjvjt\" (UniqueName: \"kubernetes.io/projected/9678ae86-ae81-4ade-8164-8f491017c3d2-kube-api-access-gjvjt\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:56.957190 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:56.957146    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9678ae86-ae81-4ade-8164-8f491017c3d2" path="/var/lib/kubelet/pods/9678ae86-ae81-4ade-8164-8f491017c3d2/volumes"
Sep 10 21:00:57.172547 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.172485    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84c98888f8-4jvjd" podUID="120bc529-f002-42fc-8ac6-653eacb78477" containerName="console" containerID="cri-o://5ab063546b395d2d277dcb1b4b09076968fb1ca79aa4308294c24b05f34fd7a1" gracePeriod=15
Sep 10 21:00:57.444855 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.444824    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:57.560881 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.560853    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-oauth-serving-cert\") pod \"120bc529-f002-42fc-8ac6-653eacb78477\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") "
Sep 10 21:00:57.561030 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.560922    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/120bc529-f002-42fc-8ac6-653eacb78477-console-oauth-config\") pod \"120bc529-f002-42fc-8ac6-653eacb78477\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") "
Sep 10 21:00:57.561030 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.560965    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/120bc529-f002-42fc-8ac6-653eacb78477-console-serving-cert\") pod \"120bc529-f002-42fc-8ac6-653eacb78477\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") "
Sep 10 21:00:57.561030 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.560998    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snd2l\" (UniqueName: \"kubernetes.io/projected/120bc529-f002-42fc-8ac6-653eacb78477-kube-api-access-snd2l\") pod \"120bc529-f002-42fc-8ac6-653eacb78477\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") "
Sep 10 21:00:57.561223 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.561029    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-console-config\") pod \"120bc529-f002-42fc-8ac6-653eacb78477\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") "
Sep 10 21:00:57.561223 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.561054    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-service-ca\") pod \"120bc529-f002-42fc-8ac6-653eacb78477\" (UID: \"120bc529-f002-42fc-8ac6-653eacb78477\") "
Sep 10 21:00:57.561394 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.561349    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "120bc529-f002-42fc-8ac6-653eacb78477" (UID: "120bc529-f002-42fc-8ac6-653eacb78477"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:00:57.561535 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.561465    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-service-ca" (OuterVolumeSpecName: "service-ca") pod "120bc529-f002-42fc-8ac6-653eacb78477" (UID: "120bc529-f002-42fc-8ac6-653eacb78477"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:00:57.561535 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.561497    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-console-config" (OuterVolumeSpecName: "console-config") pod "120bc529-f002-42fc-8ac6-653eacb78477" (UID: "120bc529-f002-42fc-8ac6-653eacb78477"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:00:57.563200 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.563169    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/120bc529-f002-42fc-8ac6-653eacb78477-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "120bc529-f002-42fc-8ac6-653eacb78477" (UID: "120bc529-f002-42fc-8ac6-653eacb78477"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:00:57.563200 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.563170    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/120bc529-f002-42fc-8ac6-653eacb78477-kube-api-access-snd2l" (OuterVolumeSpecName: "kube-api-access-snd2l") pod "120bc529-f002-42fc-8ac6-653eacb78477" (UID: "120bc529-f002-42fc-8ac6-653eacb78477"). InnerVolumeSpecName "kube-api-access-snd2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:00:57.563355 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.563199    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/120bc529-f002-42fc-8ac6-653eacb78477-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "120bc529-f002-42fc-8ac6-653eacb78477" (UID: "120bc529-f002-42fc-8ac6-653eacb78477"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:00:57.661640 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.661611    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snd2l\" (UniqueName: \"kubernetes.io/projected/120bc529-f002-42fc-8ac6-653eacb78477-kube-api-access-snd2l\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:57.661640 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.661637    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-console-config\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:57.661640 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.661647    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-service-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:57.661834 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.661655    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/120bc529-f002-42fc-8ac6-653eacb78477-oauth-serving-cert\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:57.661834 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.661665    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/120bc529-f002-42fc-8ac6-653eacb78477-console-oauth-config\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:57.661834 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.661673    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/120bc529-f002-42fc-8ac6-653eacb78477-console-serving-cert\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:00:57.669092 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.669045    2580 generic.go:356] "Generic (PLEG): container finished" podID="120bc529-f002-42fc-8ac6-653eacb78477" containerID="5ab063546b395d2d277dcb1b4b09076968fb1ca79aa4308294c24b05f34fd7a1" exitCode=0
Sep 10 21:00:57.669196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.669106    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c98888f8-4jvjd"
Sep 10 21:00:57.669196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.669131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c98888f8-4jvjd" event={"ID":"120bc529-f002-42fc-8ac6-653eacb78477","Type":"ContainerDied","Data":"5ab063546b395d2d277dcb1b4b09076968fb1ca79aa4308294c24b05f34fd7a1"}
Sep 10 21:00:57.669276 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.669208    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c98888f8-4jvjd" event={"ID":"120bc529-f002-42fc-8ac6-653eacb78477","Type":"ContainerDied","Data":"a1c2a432772a57c1429b0078447c9642d39f03e5d193ed8ed7e3198901360d46"}
Sep 10 21:00:57.669276 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.669228    2580 scope.go:117] "RemoveContainer" containerID="5ab063546b395d2d277dcb1b4b09076968fb1ca79aa4308294c24b05f34fd7a1"
Sep 10 21:00:57.677821 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.677806    2580 scope.go:117] "RemoveContainer" containerID="5ab063546b395d2d277dcb1b4b09076968fb1ca79aa4308294c24b05f34fd7a1"
Sep 10 21:00:57.678132 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:00:57.678107    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ab063546b395d2d277dcb1b4b09076968fb1ca79aa4308294c24b05f34fd7a1\": container with ID starting with 5ab063546b395d2d277dcb1b4b09076968fb1ca79aa4308294c24b05f34fd7a1 not found: ID does not exist" containerID="5ab063546b395d2d277dcb1b4b09076968fb1ca79aa4308294c24b05f34fd7a1"
Sep 10 21:00:57.678189 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.678144    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab063546b395d2d277dcb1b4b09076968fb1ca79aa4308294c24b05f34fd7a1"} err="failed to get container status \"5ab063546b395d2d277dcb1b4b09076968fb1ca79aa4308294c24b05f34fd7a1\": rpc error: code = NotFound desc = could not find container \"5ab063546b395d2d277dcb1b4b09076968fb1ca79aa4308294c24b05f34fd7a1\": container with ID starting with 5ab063546b395d2d277dcb1b4b09076968fb1ca79aa4308294c24b05f34fd7a1 not found: ID does not exist"
Sep 10 21:00:57.695924 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.695901    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84c98888f8-4jvjd"]
Sep 10 21:00:57.699753 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:57.699733    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84c98888f8-4jvjd"]
Sep 10 21:00:58.955310 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:00:58.955278    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="120bc529-f002-42fc-8ac6-653eacb78477" path="/var/lib/kubelet/pods/120bc529-f002-42fc-8ac6-653eacb78477/volumes"
Sep 10 21:01:00.710249 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:00.710214    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:01.338073 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:01.338029    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:01.394869 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:01.394844    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:01.457311 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:01.457278    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:08.831042 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:08.830996    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:08.932115 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:08.932077    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:08.992246 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:08.992191    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:13.955623 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:13.955584    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:14.010054 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:14.010021    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:16.972567 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:16.972539    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:29.027544 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:29.027507    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:51.232511 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:51.232462    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:51.629856 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:51.629831    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:01:56.316270 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.316232    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"]
Sep 10 21:01:56.316722 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.316689    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="120bc529-f002-42fc-8ac6-653eacb78477" containerName="console"
Sep 10 21:01:56.316722 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.316707    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="120bc529-f002-42fc-8ac6-653eacb78477" containerName="console"
Sep 10 21:01:56.316833 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.316725    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9678ae86-ae81-4ade-8164-8f491017c3d2" containerName="registry"
Sep 10 21:01:56.316833 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.316734    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9678ae86-ae81-4ade-8164-8f491017c3d2" containerName="registry"
Sep 10 21:01:56.316833 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.316805    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9678ae86-ae81-4ade-8164-8f491017c3d2" containerName="registry"
Sep 10 21:01:56.316833 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.316818    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="120bc529-f002-42fc-8ac6-653eacb78477" containerName="console"
Sep 10 21:01:56.319933 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.319912    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.322432 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.322411    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-m8m6k\""
Sep 10 21:01:56.322432 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.322426    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 10 21:01:56.322581 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.322466    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 10 21:01:56.322581 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.322411    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 10 21:01:56.322712 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.322696    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 10 21:01:56.322851 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.322834    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 10 21:01:56.329718 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.329212    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 10 21:01:56.335553 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.335529    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"]
Sep 10 21:01:56.359211 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.359190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99b5086c-47ec-4667-b239-0fd05ba7b65c-serving-certs-ca-bundle\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.359305 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.359218    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/99b5086c-47ec-4667-b239-0fd05ba7b65c-secret-telemeter-client\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.359305 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.359236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/99b5086c-47ec-4667-b239-0fd05ba7b65c-federate-client-tls\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.359376 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.359319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99b5086c-47ec-4667-b239-0fd05ba7b65c-metrics-client-ca\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.359376 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.359351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99b5086c-47ec-4667-b239-0fd05ba7b65c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.359441 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.359381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4h5q\" (UniqueName: \"kubernetes.io/projected/99b5086c-47ec-4667-b239-0fd05ba7b65c-kube-api-access-l4h5q\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.359441 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.359397    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99b5086c-47ec-4667-b239-0fd05ba7b65c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.359441 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.359424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/99b5086c-47ec-4667-b239-0fd05ba7b65c-telemeter-client-tls\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.460477 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.460452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99b5086c-47ec-4667-b239-0fd05ba7b65c-metrics-client-ca\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.460534 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.460485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99b5086c-47ec-4667-b239-0fd05ba7b65c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.460534 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.460509    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4h5q\" (UniqueName: \"kubernetes.io/projected/99b5086c-47ec-4667-b239-0fd05ba7b65c-kube-api-access-l4h5q\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.460534 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.460525    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99b5086c-47ec-4667-b239-0fd05ba7b65c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.460656 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.460551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/99b5086c-47ec-4667-b239-0fd05ba7b65c-telemeter-client-tls\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.460656 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.460590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99b5086c-47ec-4667-b239-0fd05ba7b65c-serving-certs-ca-bundle\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.460656 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.460614    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/99b5086c-47ec-4667-b239-0fd05ba7b65c-secret-telemeter-client\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.460656 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.460637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/99b5086c-47ec-4667-b239-0fd05ba7b65c-federate-client-tls\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.461294 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.461271    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99b5086c-47ec-4667-b239-0fd05ba7b65c-metrics-client-ca\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.461386 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.461352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99b5086c-47ec-4667-b239-0fd05ba7b65c-serving-certs-ca-bundle\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.461448 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.461429    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99b5086c-47ec-4667-b239-0fd05ba7b65c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.463136 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.463108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/99b5086c-47ec-4667-b239-0fd05ba7b65c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.463228 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.463137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/99b5086c-47ec-4667-b239-0fd05ba7b65c-telemeter-client-tls\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.463228 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.463184    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/99b5086c-47ec-4667-b239-0fd05ba7b65c-secret-telemeter-client\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.463302 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.463254    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/99b5086c-47ec-4667-b239-0fd05ba7b65c-federate-client-tls\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.489074 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.488605    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4h5q\" (UniqueName: \"kubernetes.io/projected/99b5086c-47ec-4667-b239-0fd05ba7b65c-kube-api-access-l4h5q\") pod \"telemeter-client-5f9d9f8445-7tj5p\" (UID: \"99b5086c-47ec-4667-b239-0fd05ba7b65c\") " pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.630913 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.630844    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"
Sep 10 21:01:56.910717 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:56.910693    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p"]
Sep 10 21:01:56.912643 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:01:56.912616    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99b5086c_47ec_4667_b239_0fd05ba7b65c.slice/crio-b0ab85f004a6c6594f91da6e9fec11b42a45e527850759c221ac7354a60010ad WatchSource:0}: Error finding container b0ab85f004a6c6594f91da6e9fec11b42a45e527850759c221ac7354a60010ad: Status 404 returned error can't find the container with id b0ab85f004a6c6594f91da6e9fec11b42a45e527850759c221ac7354a60010ad
Sep 10 21:01:57.893696 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:57.893652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p" event={"ID":"99b5086c-47ec-4667-b239-0fd05ba7b65c","Type":"ContainerStarted","Data":"b0ab85f004a6c6594f91da6e9fec11b42a45e527850759c221ac7354a60010ad"}
Sep 10 21:01:58.899661 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:58.899634    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p" event={"ID":"99b5086c-47ec-4667-b239-0fd05ba7b65c","Type":"ContainerStarted","Data":"711bccccbeeb03b0fe4405bb2e6b2e7189df2d59c8534fd2bb38f14c22dc5bf4"}
Sep 10 21:01:59.906526 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:59.906472    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p" event={"ID":"99b5086c-47ec-4667-b239-0fd05ba7b65c","Type":"ContainerStarted","Data":"251d60d3b98132b64d5f290216a3dceb3187c1afdbb4f9a6efa0c691a38fea0a"}
Sep 10 21:01:59.906526 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:59.906533    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p" event={"ID":"99b5086c-47ec-4667-b239-0fd05ba7b65c","Type":"ContainerStarted","Data":"d42d6128f2b2f77244abb4e2aa182e885ee74f300fc03c7cca0e6b9f116a6ad6"}
Sep 10 21:01:59.931283 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:01:59.931234    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5f9d9f8445-7tj5p" podStartSLOduration=2.074078288 podStartE2EDuration="3.931215161s" totalImagesPullingTime="1.857136873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:01:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:01:59.930342909 +0000 UTC m=+213.609841800" watchObservedRunningTime="2026-09-10 21:01:59.931215161 +0000 UTC m=+213.610714014"
Sep 10 21:02:00.065355 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:00.065328    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:00.116624 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:00.116601    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:06.484437 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:06.484400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:06.541580 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:06.541554    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:10.469651 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.469619    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-569bf6b96c-lqqc8"]
Sep 10 21:02:10.538834 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.538802    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-767f5449bd-sv8kg"]
Sep 10 21:02:10.542147 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.542132    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.572482 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.572461    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-767f5449bd-sv8kg"]
Sep 10 21:02:10.655592 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.655566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkkb2\" (UniqueName: \"kubernetes.io/projected/9ead99fe-f27a-4cee-b184-91c0f9820855-kube-api-access-hkkb2\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.655713 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.655615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ead99fe-f27a-4cee-b184-91c0f9820855-console-serving-cert\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.655713 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.655642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ead99fe-f27a-4cee-b184-91c0f9820855-console-oauth-config\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.655713 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.655665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-console-config\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.655713 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.655693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-trusted-ca-bundle\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.655893 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.655714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-oauth-serving-cert\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.655893 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.655790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-service-ca\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.757007 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.756948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkkb2\" (UniqueName: \"kubernetes.io/projected/9ead99fe-f27a-4cee-b184-91c0f9820855-kube-api-access-hkkb2\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.757007 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.756990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ead99fe-f27a-4cee-b184-91c0f9820855-console-serving-cert\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.757196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.757013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ead99fe-f27a-4cee-b184-91c0f9820855-console-oauth-config\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.757196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.757037    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-console-config\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.757196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.757085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-trusted-ca-bundle\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.757196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.757108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-oauth-serving-cert\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.757196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.757184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-service-ca\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.757917 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.757894    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-console-config\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.758017 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.757928    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-oauth-serving-cert\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.758017 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.757959    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-service-ca\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.758159 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.758137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-trusted-ca-bundle\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.759462 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.759435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ead99fe-f27a-4cee-b184-91c0f9820855-console-oauth-config\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.759548 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.759532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ead99fe-f27a-4cee-b184-91c0f9820855-console-serving-cert\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.765457 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.765435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkkb2\" (UniqueName: \"kubernetes.io/projected/9ead99fe-f27a-4cee-b184-91c0f9820855-kube-api-access-hkkb2\") pod \"console-767f5449bd-sv8kg\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") " pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:10.851473 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:10.851450    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:11.118895 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:11.118868    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-767f5449bd-sv8kg"]
Sep 10 21:02:11.122441 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:02:11.122419    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ead99fe_f27a_4cee_b184_91c0f9820855.slice/crio-1a8015914b51eba09f00dcd879caa48e55998e7ce3f9baf913c211b031cdc413 WatchSource:0}: Error finding container 1a8015914b51eba09f00dcd879caa48e55998e7ce3f9baf913c211b031cdc413: Status 404 returned error can't find the container with id 1a8015914b51eba09f00dcd879caa48e55998e7ce3f9baf913c211b031cdc413
Sep 10 21:02:11.952946 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:11.952911    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767f5449bd-sv8kg" event={"ID":"9ead99fe-f27a-4cee-b184-91c0f9820855","Type":"ContainerStarted","Data":"f02b940573a47e01443157c04c398d9d8a31ac898da834a0cf7f4fa3f44c26de"}
Sep 10 21:02:11.953340 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:11.952988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767f5449bd-sv8kg" event={"ID":"9ead99fe-f27a-4cee-b184-91c0f9820855","Type":"ContainerStarted","Data":"1a8015914b51eba09f00dcd879caa48e55998e7ce3f9baf913c211b031cdc413"}
Sep 10 21:02:11.971693 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:11.971643    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-767f5449bd-sv8kg" podStartSLOduration=1.9716267200000002 podStartE2EDuration="1.97162672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:02:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:02:11.96872588 +0000 UTC m=+225.648224744" watchObservedRunningTime="2026-09-10 21:02:11.97162672 +0000 UTC m=+225.651125569"
Sep 10 21:02:18.435660 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:18.435628    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:18.486689 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:18.486662    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:20.852320 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:20.852272    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:20.852717 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:20.852386    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:20.857476 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:20.857452    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:20.990355 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:20.990327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:02:21.169801 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:21.169729    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:21.272220 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:21.272185    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:21.329570 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:21.329542    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:31.339201 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:31.339167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:31.440196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:31.440174    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:33.160543 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:33.160500    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:33.260260 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:33.260227    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:02:35.489870 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.489788    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-569bf6b96c-lqqc8" podUID="4782740c-925f-4468-b14c-65a639fcd905" containerName="console" containerID="cri-o://f947f3efcc48ab4a2c2f88b51892b648d25a89627f3161ae6bd559428d4e59bf" gracePeriod=15
Sep 10 21:02:35.801936 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.801851    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:02:35.939315 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.939173    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-service-ca\") pod \"4782740c-925f-4468-b14c-65a639fcd905\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") "
Sep 10 21:02:35.939315 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.939302    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-console-config\") pod \"4782740c-925f-4468-b14c-65a639fcd905\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") "
Sep 10 21:02:35.939895 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.939393    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-trusted-ca-bundle\") pod \"4782740c-925f-4468-b14c-65a639fcd905\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") "
Sep 10 21:02:35.939895 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.939439    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4782740c-925f-4468-b14c-65a639fcd905-console-serving-cert\") pod \"4782740c-925f-4468-b14c-65a639fcd905\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") "
Sep 10 21:02:35.939895 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.939480    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4782740c-925f-4468-b14c-65a639fcd905-console-oauth-config\") pod \"4782740c-925f-4468-b14c-65a639fcd905\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") "
Sep 10 21:02:35.939895 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.939573    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-oauth-serving-cert\") pod \"4782740c-925f-4468-b14c-65a639fcd905\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") "
Sep 10 21:02:35.939895 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.939674    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqz6x\" (UniqueName: \"kubernetes.io/projected/4782740c-925f-4468-b14c-65a639fcd905-kube-api-access-pqz6x\") pod \"4782740c-925f-4468-b14c-65a639fcd905\" (UID: \"4782740c-925f-4468-b14c-65a639fcd905\") "
Sep 10 21:02:35.940210 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.940167    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4782740c-925f-4468-b14c-65a639fcd905" (UID: "4782740c-925f-4468-b14c-65a639fcd905"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:02:35.940210 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.940189    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-service-ca" (OuterVolumeSpecName: "service-ca") pod "4782740c-925f-4468-b14c-65a639fcd905" (UID: "4782740c-925f-4468-b14c-65a639fcd905"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:02:35.940306 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.940204    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-console-config" (OuterVolumeSpecName: "console-config") pod "4782740c-925f-4468-b14c-65a639fcd905" (UID: "4782740c-925f-4468-b14c-65a639fcd905"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:02:35.940306 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.940241    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4782740c-925f-4468-b14c-65a639fcd905" (UID: "4782740c-925f-4468-b14c-65a639fcd905"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:02:35.943471 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.943390    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4782740c-925f-4468-b14c-65a639fcd905-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4782740c-925f-4468-b14c-65a639fcd905" (UID: "4782740c-925f-4468-b14c-65a639fcd905"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:02:35.943471 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.943412    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4782740c-925f-4468-b14c-65a639fcd905-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4782740c-925f-4468-b14c-65a639fcd905" (UID: "4782740c-925f-4468-b14c-65a639fcd905"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:02:35.943471 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:35.943411    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4782740c-925f-4468-b14c-65a639fcd905-kube-api-access-pqz6x" (OuterVolumeSpecName: "kube-api-access-pqz6x") pod "4782740c-925f-4468-b14c-65a639fcd905" (UID: "4782740c-925f-4468-b14c-65a639fcd905"). InnerVolumeSpecName "kube-api-access-pqz6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:02:36.040876 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.040795    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-trusted-ca-bundle\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:02:36.040876 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.040862    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4782740c-925f-4468-b14c-65a639fcd905-console-serving-cert\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:02:36.040876 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.040879    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4782740c-925f-4468-b14c-65a639fcd905-console-oauth-config\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:02:36.041515 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.040896    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-oauth-serving-cert\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:02:36.041515 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.040913    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqz6x\" (UniqueName: \"kubernetes.io/projected/4782740c-925f-4468-b14c-65a639fcd905-kube-api-access-pqz6x\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:02:36.041515 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.040934    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-service-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:02:36.041515 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.040950    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4782740c-925f-4468-b14c-65a639fcd905-console-config\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:02:36.052688 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.052614    2580 generic.go:356] "Generic (PLEG): container finished" podID="4782740c-925f-4468-b14c-65a639fcd905" containerID="f947f3efcc48ab4a2c2f88b51892b648d25a89627f3161ae6bd559428d4e59bf" exitCode=0
Sep 10 21:02:36.053038 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.052718    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569bf6b96c-lqqc8"
Sep 10 21:02:36.053038 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.052736    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569bf6b96c-lqqc8" event={"ID":"4782740c-925f-4468-b14c-65a639fcd905","Type":"ContainerDied","Data":"f947f3efcc48ab4a2c2f88b51892b648d25a89627f3161ae6bd559428d4e59bf"}
Sep 10 21:02:36.053038 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.052838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569bf6b96c-lqqc8" event={"ID":"4782740c-925f-4468-b14c-65a639fcd905","Type":"ContainerDied","Data":"d331541a16fa6bc5277be17d6bae346878a40c1f64bdb0d8bd3c177a9942ddde"}
Sep 10 21:02:36.053038 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.052873    2580 scope.go:117] "RemoveContainer" containerID="f947f3efcc48ab4a2c2f88b51892b648d25a89627f3161ae6bd559428d4e59bf"
Sep 10 21:02:36.069774 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.069738    2580 scope.go:117] "RemoveContainer" containerID="f947f3efcc48ab4a2c2f88b51892b648d25a89627f3161ae6bd559428d4e59bf"
Sep 10 21:02:36.071617 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:02:36.071570    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f947f3efcc48ab4a2c2f88b51892b648d25a89627f3161ae6bd559428d4e59bf\": container with ID starting with f947f3efcc48ab4a2c2f88b51892b648d25a89627f3161ae6bd559428d4e59bf not found: ID does not exist" containerID="f947f3efcc48ab4a2c2f88b51892b648d25a89627f3161ae6bd559428d4e59bf"
Sep 10 21:02:36.071900 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.071641    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f947f3efcc48ab4a2c2f88b51892b648d25a89627f3161ae6bd559428d4e59bf"} err="failed to get container status \"f947f3efcc48ab4a2c2f88b51892b648d25a89627f3161ae6bd559428d4e59bf\": rpc error: code = NotFound desc = could not find container \"f947f3efcc48ab4a2c2f88b51892b648d25a89627f3161ae6bd559428d4e59bf\": container with ID starting with f947f3efcc48ab4a2c2f88b51892b648d25a89627f3161ae6bd559428d4e59bf not found: ID does not exist"
Sep 10 21:02:36.079865 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.079741    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-569bf6b96c-lqqc8"]
Sep 10 21:02:36.082125 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.082036    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-569bf6b96c-lqqc8"]
Sep 10 21:02:36.955495 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:02:36.955461    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4782740c-925f-4468-b14c-65a639fcd905" path="/var/lib/kubelet/pods/4782740c-925f-4468-b14c-65a639fcd905/volumes"
Sep 10 21:03:08.130690 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:08.130649    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:08.228223 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:08.228191    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:22.470145 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.470110    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:22.490419 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.490393    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"]
Sep 10 21:03:22.490688 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.490676    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4782740c-925f-4468-b14c-65a639fcd905" containerName="console"
Sep 10 21:03:22.490732 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.490690    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4782740c-925f-4468-b14c-65a639fcd905" containerName="console"
Sep 10 21:03:22.490767 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.490741    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4782740c-925f-4468-b14c-65a639fcd905" containerName="console"
Sep 10 21:03:22.493685 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.493670    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.496145 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.496129    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 21:03:22.496251 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.496130    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 10 21:03:22.497086 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.497070    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 21:03:22.497183 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.497149    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 10 21:03:22.497183 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.497161    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 21:03:22.497351 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.497193    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 10 21:03:22.497351 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.497236    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 10 21:03:22.505232 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.505210    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"]
Sep 10 21:03:22.562514 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.562489    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:22.587644 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.587617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gblhx\" (UniqueName: \"kubernetes.io/projected/403201ac-89be-4874-b094-5438af93bbec-kube-api-access-gblhx\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.587772 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.587674    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/403201ac-89be-4874-b094-5438af93bbec-ca\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.587772 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.587734    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/403201ac-89be-4874-b094-5438af93bbec-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.587876 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.587794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/403201ac-89be-4874-b094-5438af93bbec-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.587876 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.587823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/403201ac-89be-4874-b094-5438af93bbec-hub\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.587876 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.587851    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/403201ac-89be-4874-b094-5438af93bbec-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.637466 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.637443    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:22.688624 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.688598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gblhx\" (UniqueName: \"kubernetes.io/projected/403201ac-89be-4874-b094-5438af93bbec-kube-api-access-gblhx\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.688724 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.688645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/403201ac-89be-4874-b094-5438af93bbec-ca\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.688724 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.688679    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/403201ac-89be-4874-b094-5438af93bbec-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.688724 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.688718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/403201ac-89be-4874-b094-5438af93bbec-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.688891 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.688749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/403201ac-89be-4874-b094-5438af93bbec-hub\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.688891 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.688777    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/403201ac-89be-4874-b094-5438af93bbec-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.689529 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.689495    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/403201ac-89be-4874-b094-5438af93bbec-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.691323 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.691252    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/403201ac-89be-4874-b094-5438af93bbec-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.691323 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.691329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/403201ac-89be-4874-b094-5438af93bbec-ca\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.691323 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.691335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/403201ac-89be-4874-b094-5438af93bbec-hub\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.691746 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.691454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/403201ac-89be-4874-b094-5438af93bbec-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.697730 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.697711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gblhx\" (UniqueName: \"kubernetes.io/projected/403201ac-89be-4874-b094-5438af93bbec-kube-api-access-gblhx\") pod \"cluster-proxy-proxy-agent-6d95959c67-4w766\" (UID: \"403201ac-89be-4874-b094-5438af93bbec\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:22.817292 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:22.817273    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"
Sep 10 21:03:23.086975 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:23.086903    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766"]
Sep 10 21:03:23.089692 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:03:23.089665    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod403201ac_89be_4874_b094_5438af93bbec.slice/crio-05f74869150bb8ea9d0f528633a31722575a6eb6176b470c34c59c1edd77de2c WatchSource:0}: Error finding container 05f74869150bb8ea9d0f528633a31722575a6eb6176b470c34c59c1edd77de2c: Status 404 returned error can't find the container with id 05f74869150bb8ea9d0f528633a31722575a6eb6176b470c34c59c1edd77de2c
Sep 10 21:03:23.229387 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:23.229355    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766" event={"ID":"403201ac-89be-4874-b094-5438af93bbec","Type":"ContainerStarted","Data":"05f74869150bb8ea9d0f528633a31722575a6eb6176b470c34c59c1edd77de2c"}
Sep 10 21:03:26.873572 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:26.873503    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 21:03:27.250545 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:27.250466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766" event={"ID":"403201ac-89be-4874-b094-5438af93bbec","Type":"ContainerStarted","Data":"75e05b20e24b66a4bbfd9605abdb19c096bf647e237436bfb02204dfb295563b"}
Sep 10 21:03:27.250545 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:27.250506    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766" event={"ID":"403201ac-89be-4874-b094-5438af93bbec","Type":"ContainerStarted","Data":"b979ee40068a5a4951302296c9250dc5ddfc0e17c39024edab3719c1610dc4b7"}
Sep 10 21:03:27.250545 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:27.250517    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766" event={"ID":"403201ac-89be-4874-b094-5438af93bbec","Type":"ContainerStarted","Data":"3c995ea50cfb173846f1a0d8c515c4199222c41ea5224a2ccf1b743c3fecac99"}
Sep 10 21:03:27.272790 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:27.272739    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d95959c67-4w766" podStartSLOduration=1.8202222670000001 podStartE2EDuration="5.272724893s" totalImagesPullingTime="3.452502626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:03:27.271206612 +0000 UTC m=+300.950705485" watchObservedRunningTime="2026-09-10 21:03:27.272724893 +0000 UTC m=+300.952223742"
Sep 10 21:03:34.595447 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:34.595411    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:35.982205 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:35.982168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:38.207735 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:38.207698    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:38.311505 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:38.311470    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:39.505720 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:39.505686    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:39.607034 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:39.607000    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:40.095800 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.095766    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vnj7v"]
Sep 10 21:03:40.099053 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.099033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-vnj7v"
Sep 10 21:03:40.101659 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.101638    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 21:03:40.102694 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.102669    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 21:03:40.102785 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.102669    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7sltm\""
Sep 10 21:03:40.111692 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.111667    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vnj7v"]
Sep 10 21:03:40.216980 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.216951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a85c1168-07e6-4ed8-abf0-f94f964858aa-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vnj7v\" (UID: \"a85c1168-07e6-4ed8-abf0-f94f964858aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vnj7v"
Sep 10 21:03:40.217112 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.217016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj6ls\" (UniqueName: \"kubernetes.io/projected/a85c1168-07e6-4ed8-abf0-f94f964858aa-kube-api-access-xj6ls\") pod \"cert-manager-webhook-6486ffd997-vnj7v\" (UID: \"a85c1168-07e6-4ed8-abf0-f94f964858aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vnj7v"
Sep 10 21:03:40.217640 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.217608    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:40.317731 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.317711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a85c1168-07e6-4ed8-abf0-f94f964858aa-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vnj7v\" (UID: \"a85c1168-07e6-4ed8-abf0-f94f964858aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vnj7v"
Sep 10 21:03:40.317845 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.317753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj6ls\" (UniqueName: \"kubernetes.io/projected/a85c1168-07e6-4ed8-abf0-f94f964858aa-kube-api-access-xj6ls\") pod \"cert-manager-webhook-6486ffd997-vnj7v\" (UID: \"a85c1168-07e6-4ed8-abf0-f94f964858aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vnj7v"
Sep 10 21:03:40.321179 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.321157    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:40.342025 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.342002    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a85c1168-07e6-4ed8-abf0-f94f964858aa-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-vnj7v\" (UID: \"a85c1168-07e6-4ed8-abf0-f94f964858aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vnj7v"
Sep 10 21:03:40.342141 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.342125    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj6ls\" (UniqueName: \"kubernetes.io/projected/a85c1168-07e6-4ed8-abf0-f94f964858aa-kube-api-access-xj6ls\") pod \"cert-manager-webhook-6486ffd997-vnj7v\" (UID: \"a85c1168-07e6-4ed8-abf0-f94f964858aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-vnj7v"
Sep 10 21:03:40.408159 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.408109    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-vnj7v"
Sep 10 21:03:40.681377 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.681351    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-vnj7v"]
Sep 10 21:03:40.683686 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:03:40.683651    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda85c1168_07e6_4ed8_abf0_f94f964858aa.slice/crio-2f73eb7c53331096e72d2b3c2516784cea823088c8abb88ca1a258415ba3f438 WatchSource:0}: Error finding container 2f73eb7c53331096e72d2b3c2516784cea823088c8abb88ca1a258415ba3f438: Status 404 returned error can't find the container with id 2f73eb7c53331096e72d2b3c2516784cea823088c8abb88ca1a258415ba3f438
Sep 10 21:03:40.685592 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:40.685573    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:03:41.303765 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:41.303725    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-vnj7v" event={"ID":"a85c1168-07e6-4ed8-abf0-f94f964858aa","Type":"ContainerStarted","Data":"2f73eb7c53331096e72d2b3c2516784cea823088c8abb88ca1a258415ba3f438"}
Sep 10 21:03:41.722775 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:41.722693    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:41.923733 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:41.923663    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:44.318772 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:44.318737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-vnj7v" event={"ID":"a85c1168-07e6-4ed8-abf0-f94f964858aa","Type":"ContainerStarted","Data":"c3b44bbb8a1b8751b8d7260965ae2abe7b6f2d47aef4ba2f95f038d26333f4b8"}
Sep 10 21:03:44.319275 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:44.318818    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-vnj7v"
Sep 10 21:03:44.341858 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:44.341808    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-vnj7v" podStartSLOduration=1.2564992369999999 podStartE2EDuration="4.34179245s" totalImagesPullingTime="3.085293213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:03:44.341734494 +0000 UTC m=+318.021233344" watchObservedRunningTime="2026-09-10 21:03:44.34179245 +0000 UTC m=+318.021291300"
Sep 10 21:03:45.202610 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:45.202573    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:45.257511 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:45.257485    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:49.597663 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:49.597623    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:49.650551 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:49.650520    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:49.721213 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:49.721180    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:50.327732 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:50.327697    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-vnj7v"
Sep 10 21:03:51.612948 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:51.612909    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:51.713787 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:51.713752    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:53.636235 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:53.636199    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc"]
Sep 10 21:03:53.639567 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:53.639546    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc"
Sep 10 21:03:53.642130 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:53.642111    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:03:53.642130 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:53.642123    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-7qrj6\""
Sep 10 21:03:53.642993 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:53.642977    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 21:03:53.647745 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:53.647725    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc"]
Sep 10 21:03:53.734946 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:53.734905    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9sfg\" (UniqueName: \"kubernetes.io/projected/f4609476-6b4e-4675-8e83-2a135d5c5801-kube-api-access-f9sfg\") pod \"openshift-lws-operator-fd8ccff4c-2cvzc\" (UID: \"f4609476-6b4e-4675-8e83-2a135d5c5801\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc"
Sep 10 21:03:53.735113 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:53.735008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4609476-6b4e-4675-8e83-2a135d5c5801-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2cvzc\" (UID: \"f4609476-6b4e-4675-8e83-2a135d5c5801\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc"
Sep 10 21:03:53.836231 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:53.836195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4609476-6b4e-4675-8e83-2a135d5c5801-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2cvzc\" (UID: \"f4609476-6b4e-4675-8e83-2a135d5c5801\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc"
Sep 10 21:03:53.836352 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:53.836255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9sfg\" (UniqueName: \"kubernetes.io/projected/f4609476-6b4e-4675-8e83-2a135d5c5801-kube-api-access-f9sfg\") pod \"openshift-lws-operator-fd8ccff4c-2cvzc\" (UID: \"f4609476-6b4e-4675-8e83-2a135d5c5801\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc"
Sep 10 21:03:53.836663 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:53.836643    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4609476-6b4e-4675-8e83-2a135d5c5801-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2cvzc\" (UID: \"f4609476-6b4e-4675-8e83-2a135d5c5801\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc"
Sep 10 21:03:53.845432 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:53.845411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9sfg\" (UniqueName: \"kubernetes.io/projected/f4609476-6b4e-4675-8e83-2a135d5c5801-kube-api-access-f9sfg\") pod \"openshift-lws-operator-fd8ccff4c-2cvzc\" (UID: \"f4609476-6b4e-4675-8e83-2a135d5c5801\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc"
Sep 10 21:03:53.949620 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:53.949531    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc"
Sep 10 21:03:54.219410 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:54.219386    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc"]
Sep 10 21:03:54.222001 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:03:54.221973    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4609476_6b4e_4675_8e83_2a135d5c5801.slice/crio-5abf90a1d62459f4ed7623bdaa24373059f89b9776ad6f84f2bcc885575af3c6 WatchSource:0}: Error finding container 5abf90a1d62459f4ed7623bdaa24373059f89b9776ad6f84f2bcc885575af3c6: Status 404 returned error can't find the container with id 5abf90a1d62459f4ed7623bdaa24373059f89b9776ad6f84f2bcc885575af3c6
Sep 10 21:03:54.360153 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:54.360117    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc" event={"ID":"f4609476-6b4e-4675-8e83-2a135d5c5801","Type":"ContainerStarted","Data":"5abf90a1d62459f4ed7623bdaa24373059f89b9776ad6f84f2bcc885575af3c6"}
Sep 10 21:03:57.376608 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:57.376569    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc" event={"ID":"f4609476-6b4e-4675-8e83-2a135d5c5801","Type":"ContainerStarted","Data":"ee9b690fd03dbf6ba7ba7e41b7939e9033465a91e3acc41d253d78ea773d3a88"}
Sep 10 21:03:57.394722 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:57.394659    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2cvzc" podStartSLOduration=1.9766865980000001 podStartE2EDuration="4.394639883s" totalImagesPullingTime="2.417953285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:03:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:03:57.393181661 +0000 UTC m=+331.072680536" watchObservedRunningTime="2026-09-10 21:03:57.394639883 +0000 UTC m=+331.074138735"
Sep 10 21:03:58.055008 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:58.054973    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:58.160223 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:58.160177    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:03:59.815745 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:03:59.815713    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:09.065382 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:09.065349    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:11.148141 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:11.148108    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:12.152618 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:12.152579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:17.866470 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:17.866436    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:17.968430 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:17.968402    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:18.381232 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.381198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:18.600983 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.600955    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:18.704730 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.704654    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:18.754832 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.754807    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"]
Sep 10 21:04:18.762580 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.762562    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.765486 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.765460    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 21:04:18.765582 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.765565    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cv4zb\""
Sep 10 21:04:18.765829 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.765811    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 21:04:18.765882 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.765814    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 21:04:18.765882 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.765876    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 21:04:18.774012 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.773992    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"]
Sep 10 21:04:18.833294 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.833271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d11a2253-17f3-49f6-a19f-50b3be19957a-webhook-cert\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.833391 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.833316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g65bp\" (UniqueName: \"kubernetes.io/projected/d11a2253-17f3-49f6-a19f-50b3be19957a-kube-api-access-g65bp\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.833391 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.833373    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d11a2253-17f3-49f6-a19f-50b3be19957a-manifests\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.833466 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.833421    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d11a2253-17f3-49f6-a19f-50b3be19957a-tmp\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.833466 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.833451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d11a2253-17f3-49f6-a19f-50b3be19957a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.888725 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.888702    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:18.934006 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.933981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d11a2253-17f3-49f6-a19f-50b3be19957a-webhook-cert\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.934111 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.934024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g65bp\" (UniqueName: \"kubernetes.io/projected/d11a2253-17f3-49f6-a19f-50b3be19957a-kube-api-access-g65bp\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.934111 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.934054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d11a2253-17f3-49f6-a19f-50b3be19957a-manifests\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.934203 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.934107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d11a2253-17f3-49f6-a19f-50b3be19957a-tmp\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.934203 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.934137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d11a2253-17f3-49f6-a19f-50b3be19957a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.934462 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.934443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d11a2253-17f3-49f6-a19f-50b3be19957a-manifests\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.936230 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.936209    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d11a2253-17f3-49f6-a19f-50b3be19957a-tmp\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.936464 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.936443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d11a2253-17f3-49f6-a19f-50b3be19957a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.936536 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.936523    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d11a2253-17f3-49f6-a19f-50b3be19957a-webhook-cert\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.945134 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.945116    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g65bp\" (UniqueName: \"kubernetes.io/projected/d11a2253-17f3-49f6-a19f-50b3be19957a-kube-api-access-g65bp\") pod \"opendatahub-operator-controller-manager-d67fdbbcb-xt6xt\" (UID: \"d11a2253-17f3-49f6-a19f-50b3be19957a\") " pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:18.994112 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:18.993984    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:19.058293 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:19.058273    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:19.073293 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:19.073268    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:19.158388 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:19.158360    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:19.258659 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:19.258581    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:19.341354 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:19.341310    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:19.356793 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:19.356774    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"]
Sep 10 21:04:19.357751 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:04:19.357729    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd11a2253_17f3_49f6_a19f_50b3be19957a.slice/crio-cb27c87707df2650aae7dfa325ebe0917d2ccc39a1eeeda581f9c142c7e8f553 WatchSource:0}: Error finding container cb27c87707df2650aae7dfa325ebe0917d2ccc39a1eeeda581f9c142c7e8f553: Status 404 returned error can't find the container with id cb27c87707df2650aae7dfa325ebe0917d2ccc39a1eeeda581f9c142c7e8f553
Sep 10 21:04:19.402204 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:19.402181    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:19.460108 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:19.460081    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt" event={"ID":"d11a2253-17f3-49f6-a19f-50b3be19957a","Type":"ContainerStarted","Data":"cb27c87707df2650aae7dfa325ebe0917d2ccc39a1eeeda581f9c142c7e8f553"}
Sep 10 21:04:22.099621 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:22.099583    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:22.193867 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:22.193836    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:22.478227 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:22.478199    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt" event={"ID":"d11a2253-17f3-49f6-a19f-50b3be19957a","Type":"ContainerStarted","Data":"08be8bc8afa9485b2222a3cc840be9804f05ba6f009754c2f32ba9a8fca1b71d"}
Sep 10 21:04:24.487966 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:24.487931    2580 generic.go:356] "Generic (PLEG): container finished" podID="d11a2253-17f3-49f6-a19f-50b3be19957a" containerID="08be8bc8afa9485b2222a3cc840be9804f05ba6f009754c2f32ba9a8fca1b71d" exitCode=0
Sep 10 21:04:24.488457 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:24.487982    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt" event={"ID":"d11a2253-17f3-49f6-a19f-50b3be19957a","Type":"ContainerDied","Data":"08be8bc8afa9485b2222a3cc840be9804f05ba6f009754c2f32ba9a8fca1b71d"}
Sep 10 21:04:25.494527 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:25.494492    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt" event={"ID":"d11a2253-17f3-49f6-a19f-50b3be19957a","Type":"ContainerStarted","Data":"b5b2dedab2d4126919fbc05cbd22fce67ec65c58a19c2cc1c6222a904efcab23"}
Sep 10 21:04:25.494920 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:25.494698    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:25.518719 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:25.518669    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt" podStartSLOduration=3.045153581 podStartE2EDuration="7.518656122s" totalImagesPullingTime="3.41084337s" totalInitContainerRuntime="1.062659171s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:04:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:04:25.517569701 +0000 UTC m=+359.197068550" watchObservedRunningTime="2026-09-10 21:04:25.518656122 +0000 UTC m=+359.198155055"
Sep 10 21:04:36.500727 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:36.500699    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d67fdbbcb-xt6xt"
Sep 10 21:04:38.187415 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:38.187378    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:38.369328 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:38.369300    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:43.881048 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:43.881014    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:43.932075 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:43.932020    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"]
Sep 10 21:04:43.943849 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:43.943819    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"
Sep 10 21:04:43.947000 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:43.946981    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 21:04:43.948044 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:43.948024    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ssw74\""
Sep 10 21:04:43.948044 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:43.948037    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 21:04:43.948197 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:43.948052    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 21:04:43.948403 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:43.948390    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 21:04:43.955115 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:43.955091    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"]
Sep 10 21:04:43.963162 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:43.963138    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:43.997264 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:43.997239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t44hf\" (UniqueName: \"kubernetes.io/projected/2560a72a-01b9-4cca-898a-cef77012090f-kube-api-access-t44hf\") pod \"kube-auth-proxy-54c6c477f9-l8jqv\" (UID: \"2560a72a-01b9-4cca-898a-cef77012090f\") " pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"
Sep 10 21:04:43.997462 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:43.997437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2560a72a-01b9-4cca-898a-cef77012090f-tls-certs\") pod \"kube-auth-proxy-54c6c477f9-l8jqv\" (UID: \"2560a72a-01b9-4cca-898a-cef77012090f\") " pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"
Sep 10 21:04:43.997581 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:43.997474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2560a72a-01b9-4cca-898a-cef77012090f-tmp\") pod \"kube-auth-proxy-54c6c477f9-l8jqv\" (UID: \"2560a72a-01b9-4cca-898a-cef77012090f\") " pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"
Sep 10 21:04:44.098361 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:44.098342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2560a72a-01b9-4cca-898a-cef77012090f-tmp\") pod \"kube-auth-proxy-54c6c477f9-l8jqv\" (UID: \"2560a72a-01b9-4cca-898a-cef77012090f\") " pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"
Sep 10 21:04:44.098458 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:44.098370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t44hf\" (UniqueName: \"kubernetes.io/projected/2560a72a-01b9-4cca-898a-cef77012090f-kube-api-access-t44hf\") pod \"kube-auth-proxy-54c6c477f9-l8jqv\" (UID: \"2560a72a-01b9-4cca-898a-cef77012090f\") " pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"
Sep 10 21:04:44.098458 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:44.098421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2560a72a-01b9-4cca-898a-cef77012090f-tls-certs\") pod \"kube-auth-proxy-54c6c477f9-l8jqv\" (UID: \"2560a72a-01b9-4cca-898a-cef77012090f\") " pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"
Sep 10 21:04:44.100584 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:44.100557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2560a72a-01b9-4cca-898a-cef77012090f-tmp\") pod \"kube-auth-proxy-54c6c477f9-l8jqv\" (UID: \"2560a72a-01b9-4cca-898a-cef77012090f\") " pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"
Sep 10 21:04:44.100747 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:44.100732    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2560a72a-01b9-4cca-898a-cef77012090f-tls-certs\") pod \"kube-auth-proxy-54c6c477f9-l8jqv\" (UID: \"2560a72a-01b9-4cca-898a-cef77012090f\") " pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"
Sep 10 21:04:44.107211 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:44.107182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t44hf\" (UniqueName: \"kubernetes.io/projected/2560a72a-01b9-4cca-898a-cef77012090f-kube-api-access-t44hf\") pod \"kube-auth-proxy-54c6c477f9-l8jqv\" (UID: \"2560a72a-01b9-4cca-898a-cef77012090f\") " pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"
Sep 10 21:04:44.254601 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:44.254545    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"
Sep 10 21:04:44.526904 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:44.526811    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv"]
Sep 10 21:04:44.529581 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:04:44.529554    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2560a72a_01b9_4cca_898a_cef77012090f.slice/crio-1c95773f13bad006be6dfdcd7670cca1da84e18a76ba42ba6f85860828ce0aab WatchSource:0}: Error finding container 1c95773f13bad006be6dfdcd7670cca1da84e18a76ba42ba6f85860828ce0aab: Status 404 returned error can't find the container with id 1c95773f13bad006be6dfdcd7670cca1da84e18a76ba42ba6f85860828ce0aab
Sep 10 21:04:44.580303 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:44.580264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv" event={"ID":"2560a72a-01b9-4cca-898a-cef77012090f","Type":"ContainerStarted","Data":"1c95773f13bad006be6dfdcd7670cca1da84e18a76ba42ba6f85860828ce0aab"}
Sep 10 21:04:45.058780 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:45.058739    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:45.128988 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:45.128963    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:46.077467 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:46.077436    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:47.595734 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:47.595697    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv" event={"ID":"2560a72a-01b9-4cca-898a-cef77012090f","Type":"ContainerStarted","Data":"7a7f610928f9ce08bfd8ece16e831edcae9f59cb0f247f1a05a3afd1e34019c5"}
Sep 10 21:04:47.616651 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:47.616607    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-54c6c477f9-l8jqv" podStartSLOduration=1.913399633 podStartE2EDuration="4.616592837s" totalImagesPullingTime="2.703193204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:04:47.615236998 +0000 UTC m=+381.294735849" watchObservedRunningTime="2026-09-10 21:04:47.616592837 +0000 UTC m=+381.296091687"
Sep 10 21:04:49.381769 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:49.381741    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:49.474185 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:49.474154    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:49.573910 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:49.573872    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:49.674238 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:49.674129    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:49.888987 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:49.888949    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:49.893113 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:49.893083    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fwhcj"]
Sep 10 21:04:49.896446 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:49.896425    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj"
Sep 10 21:04:49.899088 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:49.899067    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 10 21:04:49.899178 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:49.899072    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-bn4wv\""
Sep 10 21:04:49.906700 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:49.906681    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fwhcj"]
Sep 10 21:04:49.950828 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:49.950770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzgvf\" (UniqueName: \"kubernetes.io/projected/355380ae-4412-4f96-a5f1-1e583dcfc2b1-kube-api-access-lzgvf\") pod \"model-serving-api-54d89bd5fc-fwhcj\" (UID: \"355380ae-4412-4f96-a5f1-1e583dcfc2b1\") " pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj"
Sep 10 21:04:49.950828 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:49.950810    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/355380ae-4412-4f96-a5f1-1e583dcfc2b1-tls-certs\") pod \"model-serving-api-54d89bd5fc-fwhcj\" (UID: \"355380ae-4412-4f96-a5f1-1e583dcfc2b1\") " pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj"
Sep 10 21:04:50.051953 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:50.051925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzgvf\" (UniqueName: \"kubernetes.io/projected/355380ae-4412-4f96-a5f1-1e583dcfc2b1-kube-api-access-lzgvf\") pod \"model-serving-api-54d89bd5fc-fwhcj\" (UID: \"355380ae-4412-4f96-a5f1-1e583dcfc2b1\") " pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj"
Sep 10 21:04:50.052040 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:50.051975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/355380ae-4412-4f96-a5f1-1e583dcfc2b1-tls-certs\") pod \"model-serving-api-54d89bd5fc-fwhcj\" (UID: \"355380ae-4412-4f96-a5f1-1e583dcfc2b1\") " pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj"
Sep 10 21:04:50.052144 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:04:50.052128    2580 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 10 21:04:50.052203 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:04:50.052192    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/355380ae-4412-4f96-a5f1-1e583dcfc2b1-tls-certs podName:355380ae-4412-4f96-a5f1-1e583dcfc2b1 nodeName:}" failed. No retries permitted until 2026-09-10 21:04:50.552173516 +0000 UTC m=+384.231672344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/355380ae-4412-4f96-a5f1-1e583dcfc2b1-tls-certs") pod "model-serving-api-54d89bd5fc-fwhcj" (UID: "355380ae-4412-4f96-a5f1-1e583dcfc2b1") : secret "model-serving-api-tls" not found
Sep 10 21:04:50.065415 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:50.065394    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzgvf\" (UniqueName: \"kubernetes.io/projected/355380ae-4412-4f96-a5f1-1e583dcfc2b1-kube-api-access-lzgvf\") pod \"model-serving-api-54d89bd5fc-fwhcj\" (UID: \"355380ae-4412-4f96-a5f1-1e583dcfc2b1\") " pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj"
Sep 10 21:04:50.214144 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:50.214043    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:50.312137 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:50.312104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:50.460952 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:50.460918    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:50.555369 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:50.555344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/355380ae-4412-4f96-a5f1-1e583dcfc2b1-tls-certs\") pod \"model-serving-api-54d89bd5fc-fwhcj\" (UID: \"355380ae-4412-4f96-a5f1-1e583dcfc2b1\") " pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj"
Sep 10 21:04:50.557647 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:50.557627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/355380ae-4412-4f96-a5f1-1e583dcfc2b1-tls-certs\") pod \"model-serving-api-54d89bd5fc-fwhcj\" (UID: \"355380ae-4412-4f96-a5f1-1e583dcfc2b1\") " pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj"
Sep 10 21:04:50.561700 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:50.561679    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:50.807038 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:50.806968    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj"
Sep 10 21:04:51.075359 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:51.075285    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fwhcj"]
Sep 10 21:04:51.078264 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:04:51.078236    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod355380ae_4412_4f96_a5f1_1e583dcfc2b1.slice/crio-6f7879da74ae1708304a1fda8acad6626894d14ca024ed966ee1cd88326014ed WatchSource:0}: Error finding container 6f7879da74ae1708304a1fda8acad6626894d14ca024ed966ee1cd88326014ed: Status 404 returned error can't find the container with id 6f7879da74ae1708304a1fda8acad6626894d14ca024ed966ee1cd88326014ed
Sep 10 21:04:51.614370 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:51.614335    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj" event={"ID":"355380ae-4412-4f96-a5f1-1e583dcfc2b1","Type":"ContainerStarted","Data":"6f7879da74ae1708304a1fda8acad6626894d14ca024ed966ee1cd88326014ed"}
Sep 10 21:04:52.620199 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:52.620175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj" event={"ID":"355380ae-4412-4f96-a5f1-1e583dcfc2b1","Type":"ContainerStarted","Data":"ecf19f6a2bde6d5166775affd43fa5250ad721c33a8e9cb71a381c7a0577ca69"}
Sep 10 21:04:52.620489 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:52.620298    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj"
Sep 10 21:04:52.638018 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:52.637968    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj" podStartSLOduration=2.190291568 podStartE2EDuration="3.637955306s" totalImagesPullingTime="1.447663738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:04:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:04:52.636265376 +0000 UTC m=+386.315764261" watchObservedRunningTime="2026-09-10 21:04:52.637955306 +0000 UTC m=+386.317454156"
Sep 10 21:04:57.278513 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.278478    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"]
Sep 10 21:04:57.288715 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.288691    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.291614 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.291587    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 21:04:57.291740 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.291696    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 21:04:57.292027 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.292007    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 21:04:57.293989 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.292498    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cfgz2\""
Sep 10 21:04:57.293989 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.292713    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 21:04:57.294204 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.294163    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"]
Sep 10 21:04:57.406860 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.406821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.407019 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.406867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.407019 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.406943    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.407019 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.406966    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.407019 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.406983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdfjw\" (UniqueName: \"kubernetes.io/projected/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-kube-api-access-fdfjw\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.407184 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.407035    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.407184 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.407130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.508379 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.508343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.508379 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.508382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.508623 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.508416    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.508623 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.508439    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.508623 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.508481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.508623 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.508498    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.508623 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.508519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdfjw\" (UniqueName: \"kubernetes.io/projected/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-kube-api-access-fdfjw\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.509425 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.509394    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.510856 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.510829    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.510938 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.510903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.510980 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.510927    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.510980 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.510943    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.517603 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.517579    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.517886 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.517867    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdfjw\" (UniqueName: \"kubernetes.io/projected/3739b53b-8d76-48e0-9a2b-93b4d9d2445d-kube-api-access-fdfjw\") pod \"istiod-openshift-gateway-5677ccb49b-lt9n9\" (UID: \"3739b53b-8d76-48e0-9a2b-93b4d9d2445d\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.601502 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.601472    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:04:57.881180 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:57.881154    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"]
Sep 10 21:04:57.883046 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:04:57.883018    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3739b53b_8d76_48e0_9a2b_93b4d9d2445d.slice/crio-e1cfd1840f39f29af7ab3ecb73c3d5397488dfa890ac9cfde24e323d17c341e1 WatchSource:0}: Error finding container e1cfd1840f39f29af7ab3ecb73c3d5397488dfa890ac9cfde24e323d17c341e1: Status 404 returned error can't find the container with id e1cfd1840f39f29af7ab3ecb73c3d5397488dfa890ac9cfde24e323d17c341e1
Sep 10 21:04:58.654150 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:58.654115    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9" event={"ID":"3739b53b-8d76-48e0-9a2b-93b4d9d2445d","Type":"ContainerStarted","Data":"e1cfd1840f39f29af7ab3ecb73c3d5397488dfa890ac9cfde24e323d17c341e1"}
Sep 10 21:04:59.788906 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.788773    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:59.841983 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.841947    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-chfbm"]
Sep 10 21:04:59.849360 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.849336    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chfbm"
Sep 10 21:04:59.851795 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.851772    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4"]
Sep 10 21:04:59.851938 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.851896    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-ww2mr\""
Sep 10 21:04:59.853260 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.852595    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 10 21:04:59.855852 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.855821    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-chfbm"]
Sep 10 21:04:59.855959 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.855939    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4"
Sep 10 21:04:59.858668 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.858341    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 10 21:04:59.858668 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.858576    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-686hs\""
Sep 10 21:04:59.867551 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.867532    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4"]
Sep 10 21:04:59.875898 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.875877    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:04:59.930853 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.930828    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqlmb\" (UniqueName: \"kubernetes.io/projected/990f462f-3528-4812-823b-eb6b8fd4764a-kube-api-access-xqlmb\") pod \"kserve-controller-manager-d6f4b65b8-chfbm\" (UID: \"990f462f-3528-4812-823b-eb6b8fd4764a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chfbm"
Sep 10 21:04:59.930981 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.930877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/990f462f-3528-4812-823b-eb6b8fd4764a-cert\") pod \"kserve-controller-manager-d6f4b65b8-chfbm\" (UID: \"990f462f-3528-4812-823b-eb6b8fd4764a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chfbm"
Sep 10 21:04:59.930981 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.930904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s9mq\" (UniqueName: \"kubernetes.io/projected/0da2211d-2f56-446f-991e-76bb767e7599-kube-api-access-2s9mq\") pod \"llmisvc-controller-manager-777c6bd7d5-kvbb4\" (UID: \"0da2211d-2f56-446f-991e-76bb767e7599\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4"
Sep 10 21:04:59.930981 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.930951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0da2211d-2f56-446f-991e-76bb767e7599-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kvbb4\" (UID: \"0da2211d-2f56-446f-991e-76bb767e7599\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4"
Sep 10 21:04:59.957095 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:04:59.957053    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:00.031655 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.031622    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/990f462f-3528-4812-823b-eb6b8fd4764a-cert\") pod \"kserve-controller-manager-d6f4b65b8-chfbm\" (UID: \"990f462f-3528-4812-823b-eb6b8fd4764a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chfbm"
Sep 10 21:05:00.032334 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.031677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s9mq\" (UniqueName: \"kubernetes.io/projected/0da2211d-2f56-446f-991e-76bb767e7599-kube-api-access-2s9mq\") pod \"llmisvc-controller-manager-777c6bd7d5-kvbb4\" (UID: \"0da2211d-2f56-446f-991e-76bb767e7599\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4"
Sep 10 21:05:00.032334 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.031705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0da2211d-2f56-446f-991e-76bb767e7599-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kvbb4\" (UID: \"0da2211d-2f56-446f-991e-76bb767e7599\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4"
Sep 10 21:05:00.032334 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.031795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqlmb\" (UniqueName: \"kubernetes.io/projected/990f462f-3528-4812-823b-eb6b8fd4764a-kube-api-access-xqlmb\") pod \"kserve-controller-manager-d6f4b65b8-chfbm\" (UID: \"990f462f-3528-4812-823b-eb6b8fd4764a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chfbm"
Sep 10 21:05:00.034906 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.034876    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/990f462f-3528-4812-823b-eb6b8fd4764a-cert\") pod \"kserve-controller-manager-d6f4b65b8-chfbm\" (UID: \"990f462f-3528-4812-823b-eb6b8fd4764a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chfbm"
Sep 10 21:05:00.035193 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.035175    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0da2211d-2f56-446f-991e-76bb767e7599-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-kvbb4\" (UID: \"0da2211d-2f56-446f-991e-76bb767e7599\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4"
Sep 10 21:05:00.055336 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.055237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqlmb\" (UniqueName: \"kubernetes.io/projected/990f462f-3528-4812-823b-eb6b8fd4764a-kube-api-access-xqlmb\") pod \"kserve-controller-manager-d6f4b65b8-chfbm\" (UID: \"990f462f-3528-4812-823b-eb6b8fd4764a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chfbm"
Sep 10 21:05:00.056393 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.056345    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s9mq\" (UniqueName: \"kubernetes.io/projected/0da2211d-2f56-446f-991e-76bb767e7599-kube-api-access-2s9mq\") pod \"llmisvc-controller-manager-777c6bd7d5-kvbb4\" (UID: \"0da2211d-2f56-446f-991e-76bb767e7599\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4"
Sep 10 21:05:00.070237 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.070174    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:00.168638 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.168612    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chfbm"
Sep 10 21:05:00.178960 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.178936    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4"
Sep 10 21:05:00.685043 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.684987    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 21:05:00.685170 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.685146    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 10 21:05:00.890612 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.890589    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-chfbm"]
Sep 10 21:05:00.893291 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:05:00.893266    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod990f462f_3528_4812_823b_eb6b8fd4764a.slice/crio-68bb6b962a41c382e812bff473713dea12ec0d6203494d5d8640d3e8652bb7b4 WatchSource:0}: Error finding container 68bb6b962a41c382e812bff473713dea12ec0d6203494d5d8640d3e8652bb7b4: Status 404 returned error can't find the container with id 68bb6b962a41c382e812bff473713dea12ec0d6203494d5d8640d3e8652bb7b4
Sep 10 21:05:00.893931 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:00.893908    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4"]
Sep 10 21:05:00.896203 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:05:00.896179    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0da2211d_2f56_446f_991e_76bb767e7599.slice/crio-b61096ec2357ac33ffbaf83d1712aa591b06db388a76ca422a0a20112da89980 WatchSource:0}: Error finding container b61096ec2357ac33ffbaf83d1712aa591b06db388a76ca422a0a20112da89980: Status 404 returned error can't find the container with id b61096ec2357ac33ffbaf83d1712aa591b06db388a76ca422a0a20112da89980
Sep 10 21:05:01.058913 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:01.058881    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:01.158402 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:01.158319    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:01.215140 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:01.215112    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:01.675578 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:01.675539    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9" event={"ID":"3739b53b-8d76-48e0-9a2b-93b4d9d2445d","Type":"ContainerStarted","Data":"9a88ad0d668c8da019d209fabb8ca5562db9db94c8f0b51419d89fe689b7a2b2"}
Sep 10 21:05:01.676071 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:01.676042    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:05:01.677363 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:01.677313    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4" event={"ID":"0da2211d-2f56-446f-991e-76bb767e7599","Type":"ContainerStarted","Data":"b61096ec2357ac33ffbaf83d1712aa591b06db388a76ca422a0a20112da89980"}
Sep 10 21:05:01.678318 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:01.678019    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-lt9n9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:05:01.678318 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:01.678088    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9" podUID="3739b53b-8d76-48e0-9a2b-93b4d9d2445d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:05:01.679097 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:01.679068    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chfbm" event={"ID":"990f462f-3528-4812-823b-eb6b8fd4764a","Type":"ContainerStarted","Data":"68bb6b962a41c382e812bff473713dea12ec0d6203494d5d8640d3e8652bb7b4"}
Sep 10 21:05:01.714360 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:01.714300    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9" podStartSLOduration=1.9149311629999999 podStartE2EDuration="4.714279026s" totalImagesPullingTime="2.799347863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:04:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:05:01.711681043 +0000 UTC m=+395.391179908" watchObservedRunningTime="2026-09-10 21:05:01.714279026 +0000 UTC m=+395.393777878"
Sep 10 21:05:02.685754 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:02.685715    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-lt9n9"
Sep 10 21:05:03.630167 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:03.630084    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-fwhcj"
Sep 10 21:05:03.693478 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:03.692805    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chfbm" event={"ID":"990f462f-3528-4812-823b-eb6b8fd4764a","Type":"ContainerStarted","Data":"4ef21f2ab3cfca54821eef148b6f73e2c55761320f039f39e0077d5be6432837"}
Sep 10 21:05:03.710649 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:03.710589    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chfbm" podStartSLOduration=2.24229666 podStartE2EDuration="4.710568393s" totalImagesPullingTime="2.468271733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:05:03.708530197 +0000 UTC m=+397.388029044" watchObservedRunningTime="2026-09-10 21:05:03.710568393 +0000 UTC m=+397.390067242"
Sep 10 21:05:04.701385 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:04.701345    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4" event={"ID":"0da2211d-2f56-446f-991e-76bb767e7599","Type":"ContainerStarted","Data":"cbda795743969cb47859511e246b72e540a5d12057b9db569b729081d2755f97"}
Sep 10 21:05:04.701719 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:04.701648    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chfbm"
Sep 10 21:05:04.701719 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:04.701693    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4"
Sep 10 21:05:04.725723 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:04.725678    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4" podStartSLOduration=2.64181273 podStartE2EDuration="5.72566398s" totalImagesPullingTime="3.08385125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:05:04.723885157 +0000 UTC m=+398.403384007" watchObservedRunningTime="2026-09-10 21:05:04.72566398 +0000 UTC m=+398.405162830"
Sep 10 21:05:10.684168 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:10.684130    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:10.780532 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:10.780492    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:11.011627 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:11.011544    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:35.708443 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:35.708407    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-kvbb4"
Sep 10 21:05:35.711727 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:35.711705    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chfbm"
Sep 10 21:05:36.418200 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:36.418168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:36.519241 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:36.519204    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:39.628202 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:39.628160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:47.417939 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:47.417898    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-767f5449bd-sv8kg"]
Sep 10 21:05:47.545908 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:47.545880    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:47.646509 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:47.646479    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:47.710808 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:47.710746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:50.393346 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:50.393261    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:05:50.486945 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:05:50.486915    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:04.410052 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:04.410016    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:06.835638 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:06.835597    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:07.231816 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:07.231738    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:08.880548 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:08.880513    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-xhb5c"]
Sep 10 21:06:08.883940 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:08.883922    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-xhb5c"
Sep 10 21:06:08.886513 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:08.886486    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 21:06:08.886625 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:08.886547    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 21:06:08.887404 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:08.887387    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-9jczr\""
Sep 10 21:06:08.894539 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:08.894519    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-xhb5c"]
Sep 10 21:06:08.988549 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:08.988523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dwx6\" (UniqueName: \"kubernetes.io/projected/60960b03-def6-4a77-8598-e098e49b5500-kube-api-access-5dwx6\") pod \"authorino-operator-76b478bd78-xhb5c\" (UID: \"60960b03-def6-4a77-8598-e098e49b5500\") " pod="kuadrant-system/authorino-operator-76b478bd78-xhb5c"
Sep 10 21:06:09.043294 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:09.043258    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:09.089369 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:09.089340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dwx6\" (UniqueName: \"kubernetes.io/projected/60960b03-def6-4a77-8598-e098e49b5500-kube-api-access-5dwx6\") pod \"authorino-operator-76b478bd78-xhb5c\" (UID: \"60960b03-def6-4a77-8598-e098e49b5500\") " pod="kuadrant-system/authorino-operator-76b478bd78-xhb5c"
Sep 10 21:06:09.100183 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:09.100157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dwx6\" (UniqueName: \"kubernetes.io/projected/60960b03-def6-4a77-8598-e098e49b5500-kube-api-access-5dwx6\") pod \"authorino-operator-76b478bd78-xhb5c\" (UID: \"60960b03-def6-4a77-8598-e098e49b5500\") " pod="kuadrant-system/authorino-operator-76b478bd78-xhb5c"
Sep 10 21:06:09.144898 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:09.144840    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:09.194384 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:09.194359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-xhb5c"
Sep 10 21:06:09.473887 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:09.473863    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-xhb5c"]
Sep 10 21:06:09.476593 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:06:09.476563    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60960b03_def6_4a77_8598_e098e49b5500.slice/crio-903d121065ca2fcc582853c49e17d511973ba211e73a48c37731c90180fae40c WatchSource:0}: Error finding container 903d121065ca2fcc582853c49e17d511973ba211e73a48c37731c90180fae40c: Status 404 returned error can't find the container with id 903d121065ca2fcc582853c49e17d511973ba211e73a48c37731c90180fae40c
Sep 10 21:06:09.591807 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:09.591782    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:09.995367 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:09.995336    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-xhb5c" event={"ID":"60960b03-def6-4a77-8598-e098e49b5500","Type":"ContainerStarted","Data":"903d121065ca2fcc582853c49e17d511973ba211e73a48c37731c90180fae40c"}
Sep 10 21:06:10.012940 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:10.012915    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:10.404962 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:10.404930    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:11.069307 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:11.069276    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:12.069743 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.069662    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:12.170134 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.170098    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:12.437684 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.437655    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-767f5449bd-sv8kg" podUID="9ead99fe-f27a-4cee-b184-91c0f9820855" containerName="console" containerID="cri-o://f02b940573a47e01443157c04c398d9d8a31ac898da834a0cf7f4fa3f44c26de" gracePeriod=15
Sep 10 21:06:12.679004 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.678982    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:06:12.822903 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.822880    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ead99fe-f27a-4cee-b184-91c0f9820855-console-oauth-config\") pod \"9ead99fe-f27a-4cee-b184-91c0f9820855\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") "
Sep 10 21:06:12.823078 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.822914    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ead99fe-f27a-4cee-b184-91c0f9820855-console-serving-cert\") pod \"9ead99fe-f27a-4cee-b184-91c0f9820855\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") "
Sep 10 21:06:12.823078 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.822941    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-console-config\") pod \"9ead99fe-f27a-4cee-b184-91c0f9820855\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") "
Sep 10 21:06:12.823078 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.822969    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-oauth-serving-cert\") pod \"9ead99fe-f27a-4cee-b184-91c0f9820855\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") "
Sep 10 21:06:12.823078 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.823022    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkkb2\" (UniqueName: \"kubernetes.io/projected/9ead99fe-f27a-4cee-b184-91c0f9820855-kube-api-access-hkkb2\") pod \"9ead99fe-f27a-4cee-b184-91c0f9820855\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") "
Sep 10 21:06:12.823285 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.823090    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-trusted-ca-bundle\") pod \"9ead99fe-f27a-4cee-b184-91c0f9820855\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") "
Sep 10 21:06:12.823285 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.823128    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-service-ca\") pod \"9ead99fe-f27a-4cee-b184-91c0f9820855\" (UID: \"9ead99fe-f27a-4cee-b184-91c0f9820855\") "
Sep 10 21:06:12.823443 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.823336    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-console-config" (OuterVolumeSpecName: "console-config") pod "9ead99fe-f27a-4cee-b184-91c0f9820855" (UID: "9ead99fe-f27a-4cee-b184-91c0f9820855"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:06:12.823554 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.823445    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9ead99fe-f27a-4cee-b184-91c0f9820855" (UID: "9ead99fe-f27a-4cee-b184-91c0f9820855"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:06:12.823554 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.823522    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9ead99fe-f27a-4cee-b184-91c0f9820855" (UID: "9ead99fe-f27a-4cee-b184-91c0f9820855"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:06:12.823663 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.823642    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-service-ca" (OuterVolumeSpecName: "service-ca") pod "9ead99fe-f27a-4cee-b184-91c0f9820855" (UID: "9ead99fe-f27a-4cee-b184-91c0f9820855"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:06:12.825015 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.824989    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ead99fe-f27a-4cee-b184-91c0f9820855-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9ead99fe-f27a-4cee-b184-91c0f9820855" (UID: "9ead99fe-f27a-4cee-b184-91c0f9820855"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:06:12.825196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.825045    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ead99fe-f27a-4cee-b184-91c0f9820855-kube-api-access-hkkb2" (OuterVolumeSpecName: "kube-api-access-hkkb2") pod "9ead99fe-f27a-4cee-b184-91c0f9820855" (UID: "9ead99fe-f27a-4cee-b184-91c0f9820855"). InnerVolumeSpecName "kube-api-access-hkkb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:06:12.825196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.825070    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ead99fe-f27a-4cee-b184-91c0f9820855-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9ead99fe-f27a-4cee-b184-91c0f9820855" (UID: "9ead99fe-f27a-4cee-b184-91c0f9820855"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:06:12.924586 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.924534    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkkb2\" (UniqueName: \"kubernetes.io/projected/9ead99fe-f27a-4cee-b184-91c0f9820855-kube-api-access-hkkb2\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:06:12.924586 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.924555    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-trusted-ca-bundle\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:06:12.924586 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.924568    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-service-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:06:12.924586 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.924576    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ead99fe-f27a-4cee-b184-91c0f9820855-console-oauth-config\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:06:12.924586 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.924585    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ead99fe-f27a-4cee-b184-91c0f9820855-console-serving-cert\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:06:12.924785 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.924594    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-console-config\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:06:12.924785 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:12.924602    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ead99fe-f27a-4cee-b184-91c0f9820855-oauth-serving-cert\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:06:13.012939 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:13.012901    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-xhb5c" event={"ID":"60960b03-def6-4a77-8598-e098e49b5500","Type":"ContainerStarted","Data":"7f60396d40d634d6faacdd6c5303fcf6a00156f65b1e06ef9b23d8e9d5f832c3"}
Sep 10 21:06:13.013041 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:13.012956    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-xhb5c"
Sep 10 21:06:13.014202 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:13.014181    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ead99fe-f27a-4cee-b184-91c0f9820855" containerID="f02b940573a47e01443157c04c398d9d8a31ac898da834a0cf7f4fa3f44c26de" exitCode=0
Sep 10 21:06:13.014302 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:13.014230    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767f5449bd-sv8kg" event={"ID":"9ead99fe-f27a-4cee-b184-91c0f9820855","Type":"ContainerDied","Data":"f02b940573a47e01443157c04c398d9d8a31ac898da834a0cf7f4fa3f44c26de"}
Sep 10 21:06:13.014302 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:13.014233    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767f5449bd-sv8kg"
Sep 10 21:06:13.014302 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:13.014259    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767f5449bd-sv8kg" event={"ID":"9ead99fe-f27a-4cee-b184-91c0f9820855","Type":"ContainerDied","Data":"1a8015914b51eba09f00dcd879caa48e55998e7ce3f9baf913c211b031cdc413"}
Sep 10 21:06:13.014302 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:13.014277    2580 scope.go:117] "RemoveContainer" containerID="f02b940573a47e01443157c04c398d9d8a31ac898da834a0cf7f4fa3f44c26de"
Sep 10 21:06:13.022161 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:13.022145    2580 scope.go:117] "RemoveContainer" containerID="f02b940573a47e01443157c04c398d9d8a31ac898da834a0cf7f4fa3f44c26de"
Sep 10 21:06:13.022404 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:06:13.022387    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f02b940573a47e01443157c04c398d9d8a31ac898da834a0cf7f4fa3f44c26de\": container with ID starting with f02b940573a47e01443157c04c398d9d8a31ac898da834a0cf7f4fa3f44c26de not found: ID does not exist" containerID="f02b940573a47e01443157c04c398d9d8a31ac898da834a0cf7f4fa3f44c26de"
Sep 10 21:06:13.022455 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:13.022411    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f02b940573a47e01443157c04c398d9d8a31ac898da834a0cf7f4fa3f44c26de"} err="failed to get container status \"f02b940573a47e01443157c04c398d9d8a31ac898da834a0cf7f4fa3f44c26de\": rpc error: code = NotFound desc = could not find container \"f02b940573a47e01443157c04c398d9d8a31ac898da834a0cf7f4fa3f44c26de\": container with ID starting with f02b940573a47e01443157c04c398d9d8a31ac898da834a0cf7f4fa3f44c26de not found: ID does not exist"
Sep 10 21:06:13.031005 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:13.030966    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-xhb5c" podStartSLOduration=2.118133092 podStartE2EDuration="5.030954311s" totalImagesPullingTime="2.912821219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:06:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:06:13.029230327 +0000 UTC m=+466.708729177" watchObservedRunningTime="2026-09-10 21:06:13.030954311 +0000 UTC m=+466.710453160"
Sep 10 21:06:13.042690 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:13.042664    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-767f5449bd-sv8kg"]
Sep 10 21:06:13.048332 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:13.048312    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-767f5449bd-sv8kg"]
Sep 10 21:06:13.827318 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:13.827289    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:14.040518 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:14.040490    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:14.956163 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:14.956132    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ead99fe-f27a-4cee-b184-91c0f9820855" path="/var/lib/kubelet/pods/9ead99fe-f27a-4cee-b184-91c0f9820855/volumes"
Sep 10 21:06:16.806567 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:16.806530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:16.908338 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:16.908305    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:20.810267 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:20.810235    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:20.913786 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:20.913754    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:21.029429 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:21.029400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:21.122394 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:21.122327    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:21.589579 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:21.589551    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:21.804454 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:21.804423    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:24.024173 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:24.024141    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-xhb5c"
Sep 10 21:06:26.017498 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:26.017465    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:26.117969 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:26.117942    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:31.767539 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:31.767503    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"]
Sep 10 21:06:31.768114 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:31.768092    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ead99fe-f27a-4cee-b184-91c0f9820855" containerName="console"
Sep 10 21:06:31.768114 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:31.768114    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ead99fe-f27a-4cee-b184-91c0f9820855" containerName="console"
Sep 10 21:06:31.768276 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:31.768218    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ead99fe-f27a-4cee-b184-91c0f9820855" containerName="console"
Sep 10 21:06:31.774898 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:31.774883    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:31.777613 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:31.777593    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9dkn7\""
Sep 10 21:06:31.786163 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:31.786142    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"]
Sep 10 21:06:31.866988 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:31.866964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5eb64398-0744-4bb9-83a8-e6c8a44cc129-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-bxmwd\" (UID: \"5eb64398-0744-4bb9-83a8-e6c8a44cc129\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:31.867121 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:31.866996    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sklj\" (UniqueName: \"kubernetes.io/projected/5eb64398-0744-4bb9-83a8-e6c8a44cc129-kube-api-access-9sklj\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-bxmwd\" (UID: \"5eb64398-0744-4bb9-83a8-e6c8a44cc129\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:31.968112 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:31.968089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5eb64398-0744-4bb9-83a8-e6c8a44cc129-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-bxmwd\" (UID: \"5eb64398-0744-4bb9-83a8-e6c8a44cc129\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:31.968112 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:31.968119    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sklj\" (UniqueName: \"kubernetes.io/projected/5eb64398-0744-4bb9-83a8-e6c8a44cc129-kube-api-access-9sklj\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-bxmwd\" (UID: \"5eb64398-0744-4bb9-83a8-e6c8a44cc129\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:31.968432 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:31.968415    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5eb64398-0744-4bb9-83a8-e6c8a44cc129-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-bxmwd\" (UID: \"5eb64398-0744-4bb9-83a8-e6c8a44cc129\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:31.977117 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:31.977090    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sklj\" (UniqueName: \"kubernetes.io/projected/5eb64398-0744-4bb9-83a8-e6c8a44cc129-kube-api-access-9sklj\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-bxmwd\" (UID: \"5eb64398-0744-4bb9-83a8-e6c8a44cc129\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:32.085275 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:32.085251    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:32.364929 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:32.364905    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"]
Sep 10 21:06:32.367339 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:06:32.367311    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb64398_0744_4bb9_83a8_e6c8a44cc129.slice/crio-cc4d5c6cc234b4728880370e06583585f8c6f64f9a91446c7e18c861c4e6cee9 WatchSource:0}: Error finding container cc4d5c6cc234b4728880370e06583585f8c6f64f9a91446c7e18c861c4e6cee9: Status 404 returned error can't find the container with id cc4d5c6cc234b4728880370e06583585f8c6f64f9a91446c7e18c861c4e6cee9
Sep 10 21:06:32.426682 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:32.426656    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:32.528269 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:32.528242    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:32.947217 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:32.947173    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"]
Sep 10 21:06:32.949533 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:32.949506    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"]
Sep 10 21:06:32.987007 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:32.986977    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"]
Sep 10 21:06:32.992731 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:32.992712    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"
Sep 10 21:06:33.009043 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:33.009015    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"]
Sep 10 21:06:33.078357 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:33.078324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec5d154d-fbb6-4508-b233-207881db4da5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vrlnt\" (UID: \"ec5d154d-fbb6-4508-b233-207881db4da5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"
Sep 10 21:06:33.078513 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:33.078386    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql42x\" (UniqueName: \"kubernetes.io/projected/ec5d154d-fbb6-4508-b233-207881db4da5-kube-api-access-ql42x\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vrlnt\" (UID: \"ec5d154d-fbb6-4508-b233-207881db4da5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"
Sep 10 21:06:33.179273 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:33.179238    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql42x\" (UniqueName: \"kubernetes.io/projected/ec5d154d-fbb6-4508-b233-207881db4da5-kube-api-access-ql42x\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vrlnt\" (UID: \"ec5d154d-fbb6-4508-b233-207881db4da5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"
Sep 10 21:06:33.179431 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:33.179377    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec5d154d-fbb6-4508-b233-207881db4da5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vrlnt\" (UID: \"ec5d154d-fbb6-4508-b233-207881db4da5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"
Sep 10 21:06:33.179744 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:33.179727    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec5d154d-fbb6-4508-b233-207881db4da5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vrlnt\" (UID: \"ec5d154d-fbb6-4508-b233-207881db4da5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"
Sep 10 21:06:33.190402 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:33.190321    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql42x\" (UniqueName: \"kubernetes.io/projected/ec5d154d-fbb6-4508-b233-207881db4da5-kube-api-access-ql42x\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-vrlnt\" (UID: \"ec5d154d-fbb6-4508-b233-207881db4da5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"
Sep 10 21:06:33.306041 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:33.306011    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"
Sep 10 21:06:33.588770 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:33.588737    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"]
Sep 10 21:06:33.591082 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:06:33.591036    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec5d154d_fbb6_4508_b233_207881db4da5.slice/crio-47c6e15723d9ed085d0a991a09c848adab6cd5adf50efae163d4273dc64f428a WatchSource:0}: Error finding container 47c6e15723d9ed085d0a991a09c848adab6cd5adf50efae163d4273dc64f428a: Status 404 returned error can't find the container with id 47c6e15723d9ed085d0a991a09c848adab6cd5adf50efae163d4273dc64f428a
Sep 10 21:06:34.119005 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:34.118975    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt" event={"ID":"ec5d154d-fbb6-4508-b233-207881db4da5","Type":"ContainerStarted","Data":"47c6e15723d9ed085d0a991a09c848adab6cd5adf50efae163d4273dc64f428a"}
Sep 10 21:06:36.808265 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:36.808231    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:36.904710 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:36.904674    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:37.136215 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:37.136129    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt" event={"ID":"ec5d154d-fbb6-4508-b233-207881db4da5","Type":"ContainerStarted","Data":"7de70a3eac1b5d6ad11a18d3c479bd53023592ff9640ee6ef08acea5f47ec355"}
Sep 10 21:06:37.136215 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:37.136202    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"
Sep 10 21:06:37.137767 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:37.137722    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd" podUID="5eb64398-0744-4bb9-83a8-e6c8a44cc129" containerName="manager" containerID="cri-o://43fb9dcfe927cac4881a7cb9aae47b1c58a8849a4bdf8f073d1714ea7bfe2f32" gracePeriod=2
Sep 10 21:06:37.156042 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:06:37.156014    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-bxmwd\" is forbidden: User \"system:node:ip-10-0-135-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-142.ec2.internal' and this object" podUID="5eb64398-0744-4bb9-83a8-e6c8a44cc129" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:37.156166 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:37.156045    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt" podStartSLOduration=2.490231197 podStartE2EDuration="5.156028713s" totalImagesPullingTime="2.665797516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:06:37.154135052 +0000 UTC m=+490.833633901" watchObservedRunningTime="2026-09-10 21:06:37.156028713 +0000 UTC m=+490.835527563"
Sep 10 21:06:37.368366 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:37.368345    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:37.370851 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:06:37.370823    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-bxmwd\" is forbidden: User \"system:node:ip-10-0-135-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-142.ec2.internal' and this object" podUID="5eb64398-0744-4bb9-83a8-e6c8a44cc129" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:37.518045 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:37.517993    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sklj\" (UniqueName: \"kubernetes.io/projected/5eb64398-0744-4bb9-83a8-e6c8a44cc129-kube-api-access-9sklj\") pod \"5eb64398-0744-4bb9-83a8-e6c8a44cc129\" (UID: \"5eb64398-0744-4bb9-83a8-e6c8a44cc129\") "
Sep 10 21:06:37.518181 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:37.518074    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5eb64398-0744-4bb9-83a8-e6c8a44cc129-extensions-socket-volume\") pod \"5eb64398-0744-4bb9-83a8-e6c8a44cc129\" (UID: \"5eb64398-0744-4bb9-83a8-e6c8a44cc129\") "
Sep 10 21:06:37.518296 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:37.518271    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb64398-0744-4bb9-83a8-e6c8a44cc129-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5eb64398-0744-4bb9-83a8-e6c8a44cc129" (UID: "5eb64398-0744-4bb9-83a8-e6c8a44cc129"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:06:37.520106 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:37.520083    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb64398-0744-4bb9-83a8-e6c8a44cc129-kube-api-access-9sklj" (OuterVolumeSpecName: "kube-api-access-9sklj") pod "5eb64398-0744-4bb9-83a8-e6c8a44cc129" (UID: "5eb64398-0744-4bb9-83a8-e6c8a44cc129"). InnerVolumeSpecName "kube-api-access-9sklj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:06:37.619365 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:37.619345    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sklj\" (UniqueName: \"kubernetes.io/projected/5eb64398-0744-4bb9-83a8-e6c8a44cc129-kube-api-access-9sklj\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:06:37.619365 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:37.619366    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5eb64398-0744-4bb9-83a8-e6c8a44cc129-extensions-socket-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:06:38.143734 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:38.143699    2580 generic.go:356] "Generic (PLEG): container finished" podID="5eb64398-0744-4bb9-83a8-e6c8a44cc129" containerID="43fb9dcfe927cac4881a7cb9aae47b1c58a8849a4bdf8f073d1714ea7bfe2f32" exitCode=2
Sep 10 21:06:38.144132 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:38.143740    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:38.144132 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:38.143804    2580 scope.go:117] "RemoveContainer" containerID="43fb9dcfe927cac4881a7cb9aae47b1c58a8849a4bdf8f073d1714ea7bfe2f32"
Sep 10 21:06:38.146151 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:06:38.146124    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-bxmwd\" is forbidden: User \"system:node:ip-10-0-135-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-142.ec2.internal' and this object" podUID="5eb64398-0744-4bb9-83a8-e6c8a44cc129" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:38.153707 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:38.153689    2580 scope.go:117] "RemoveContainer" containerID="43fb9dcfe927cac4881a7cb9aae47b1c58a8849a4bdf8f073d1714ea7bfe2f32"
Sep 10 21:06:38.153981 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:06:38.153965    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43fb9dcfe927cac4881a7cb9aae47b1c58a8849a4bdf8f073d1714ea7bfe2f32\": container with ID starting with 43fb9dcfe927cac4881a7cb9aae47b1c58a8849a4bdf8f073d1714ea7bfe2f32 not found: ID does not exist" containerID="43fb9dcfe927cac4881a7cb9aae47b1c58a8849a4bdf8f073d1714ea7bfe2f32"
Sep 10 21:06:38.154036 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:38.153988    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43fb9dcfe927cac4881a7cb9aae47b1c58a8849a4bdf8f073d1714ea7bfe2f32"} err="failed to get container status \"43fb9dcfe927cac4881a7cb9aae47b1c58a8849a4bdf8f073d1714ea7bfe2f32\": rpc error: code = NotFound desc = could not find container \"43fb9dcfe927cac4881a7cb9aae47b1c58a8849a4bdf8f073d1714ea7bfe2f32\": container with ID starting with 43fb9dcfe927cac4881a7cb9aae47b1c58a8849a4bdf8f073d1714ea7bfe2f32 not found: ID does not exist"
Sep 10 21:06:38.154560 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:06:38.154540    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-bxmwd\" is forbidden: User \"system:node:ip-10-0-135-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-142.ec2.internal' and this object" podUID="5eb64398-0744-4bb9-83a8-e6c8a44cc129" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-bxmwd"
Sep 10 21:06:38.956118 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:38.956086    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5eb64398-0744-4bb9-83a8-e6c8a44cc129" path="/var/lib/kubelet/pods/5eb64398-0744-4bb9-83a8-e6c8a44cc129/volumes"
Sep 10 21:06:40.578711 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:40.578676    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:40.629442 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:40.629409    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:40.787455 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:40.787429    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:40.840912 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:40.840845    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:46.978670 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:46.978633    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:47.034312 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:47.034277    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:47.825801 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:47.825767    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:47.924164 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:47.924133    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:48.146627 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:48.146557    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"
Sep 10 21:06:48.471873 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:48.471838    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:48.565557 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:48.565530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:54.183014 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:54.182977    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:06:54.242939 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:06:54.242913    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:00.787785 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:00.787755    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"]
Sep 10 21:07:00.788193 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:00.787961    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt" podUID="ec5d154d-fbb6-4508-b233-207881db4da5" containerName="manager" containerID="cri-o://7de70a3eac1b5d6ad11a18d3c479bd53023592ff9640ee6ef08acea5f47ec355" gracePeriod=10
Sep 10 21:07:01.048579 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.048516    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"
Sep 10 21:07:01.087691 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.087664    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql42x\" (UniqueName: \"kubernetes.io/projected/ec5d154d-fbb6-4508-b233-207881db4da5-kube-api-access-ql42x\") pod \"ec5d154d-fbb6-4508-b233-207881db4da5\" (UID: \"ec5d154d-fbb6-4508-b233-207881db4da5\") "
Sep 10 21:07:01.087866 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.087763    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec5d154d-fbb6-4508-b233-207881db4da5-extensions-socket-volume\") pod \"ec5d154d-fbb6-4508-b233-207881db4da5\" (UID: \"ec5d154d-fbb6-4508-b233-207881db4da5\") "
Sep 10 21:07:01.088226 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.088205    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec5d154d-fbb6-4508-b233-207881db4da5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ec5d154d-fbb6-4508-b233-207881db4da5" (UID: "ec5d154d-fbb6-4508-b233-207881db4da5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:07:01.089740 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.089719    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec5d154d-fbb6-4508-b233-207881db4da5-kube-api-access-ql42x" (OuterVolumeSpecName: "kube-api-access-ql42x") pod "ec5d154d-fbb6-4508-b233-207881db4da5" (UID: "ec5d154d-fbb6-4508-b233-207881db4da5"). InnerVolumeSpecName "kube-api-access-ql42x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:07:01.188540 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.188502    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec5d154d-fbb6-4508-b233-207881db4da5-extensions-socket-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:07:01.188540 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.188535    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql42x\" (UniqueName: \"kubernetes.io/projected/ec5d154d-fbb6-4508-b233-207881db4da5-kube-api-access-ql42x\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:07:01.255141 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.255111    2580 generic.go:356] "Generic (PLEG): container finished" podID="ec5d154d-fbb6-4508-b233-207881db4da5" containerID="7de70a3eac1b5d6ad11a18d3c479bd53023592ff9640ee6ef08acea5f47ec355" exitCode=0
Sep 10 21:07:01.255311 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.255152    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"
Sep 10 21:07:01.255311 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.255167    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt" event={"ID":"ec5d154d-fbb6-4508-b233-207881db4da5","Type":"ContainerDied","Data":"7de70a3eac1b5d6ad11a18d3c479bd53023592ff9640ee6ef08acea5f47ec355"}
Sep 10 21:07:01.255311 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.255209    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt" event={"ID":"ec5d154d-fbb6-4508-b233-207881db4da5","Type":"ContainerDied","Data":"47c6e15723d9ed085d0a991a09c848adab6cd5adf50efae163d4273dc64f428a"}
Sep 10 21:07:01.255311 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.255231    2580 scope.go:117] "RemoveContainer" containerID="7de70a3eac1b5d6ad11a18d3c479bd53023592ff9640ee6ef08acea5f47ec355"
Sep 10 21:07:01.263542 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.263523    2580 scope.go:117] "RemoveContainer" containerID="7de70a3eac1b5d6ad11a18d3c479bd53023592ff9640ee6ef08acea5f47ec355"
Sep 10 21:07:01.263796 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:07:01.263774    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7de70a3eac1b5d6ad11a18d3c479bd53023592ff9640ee6ef08acea5f47ec355\": container with ID starting with 7de70a3eac1b5d6ad11a18d3c479bd53023592ff9640ee6ef08acea5f47ec355 not found: ID does not exist" containerID="7de70a3eac1b5d6ad11a18d3c479bd53023592ff9640ee6ef08acea5f47ec355"
Sep 10 21:07:01.263840 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.263805    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7de70a3eac1b5d6ad11a18d3c479bd53023592ff9640ee6ef08acea5f47ec355"} err="failed to get container status \"7de70a3eac1b5d6ad11a18d3c479bd53023592ff9640ee6ef08acea5f47ec355\": rpc error: code = NotFound desc = could not find container \"7de70a3eac1b5d6ad11a18d3c479bd53023592ff9640ee6ef08acea5f47ec355\": container with ID starting with 7de70a3eac1b5d6ad11a18d3c479bd53023592ff9640ee6ef08acea5f47ec355 not found: ID does not exist"
Sep 10 21:07:01.283400 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.283376    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"]
Sep 10 21:07:01.287591 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:01.287570    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-vrlnt"]
Sep 10 21:07:02.956343 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:02.956302    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec5d154d-fbb6-4508-b233-207881db4da5" path="/var/lib/kubelet/pods/ec5d154d-fbb6-4508-b233-207881db4da5/volumes"
Sep 10 21:07:04.571323 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:04.571292    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:04.822159 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:04.822080    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:05.482895 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:05.482865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:15.471147 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:15.471106    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:15.571815 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:15.571791    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:21.369015 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.368942    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:21.493649 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.493613    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:21.596725 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.596693    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:21.743683 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.743614    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:07:21.743987 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.743972    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eb64398-0744-4bb9-83a8-e6c8a44cc129" containerName="manager"
Sep 10 21:07:21.744079 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.743988    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eb64398-0744-4bb9-83a8-e6c8a44cc129" containerName="manager"
Sep 10 21:07:21.744079 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.744009    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec5d154d-fbb6-4508-b233-207881db4da5" containerName="manager"
Sep 10 21:07:21.744079 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.744015    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec5d154d-fbb6-4508-b233-207881db4da5" containerName="manager"
Sep 10 21:07:21.744180 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.744083    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec5d154d-fbb6-4508-b233-207881db4da5" containerName="manager"
Sep 10 21:07:21.744180 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.744097    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eb64398-0744-4bb9-83a8-e6c8a44cc129" containerName="manager"
Sep 10 21:07:21.747017 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.747002    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"
Sep 10 21:07:21.749727 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.749706    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cn77b\""
Sep 10 21:07:21.749841 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.749733    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 21:07:21.758435 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.758413    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:07:21.788033 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.788007    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:07:21.842952 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.842932    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5hb8\" (UniqueName: \"kubernetes.io/projected/8b505246-b351-447c-aa7d-711bde6225e3-kube-api-access-k5hb8\") pod \"limitador-limitador-84994f7ffc-mmj7x\" (UID: \"8b505246-b351-447c-aa7d-711bde6225e3\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"
Sep 10 21:07:21.843046 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.842965    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8b505246-b351-447c-aa7d-711bde6225e3-config-file\") pod \"limitador-limitador-84994f7ffc-mmj7x\" (UID: \"8b505246-b351-447c-aa7d-711bde6225e3\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"
Sep 10 21:07:21.943489 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.943464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5hb8\" (UniqueName: \"kubernetes.io/projected/8b505246-b351-447c-aa7d-711bde6225e3-kube-api-access-k5hb8\") pod \"limitador-limitador-84994f7ffc-mmj7x\" (UID: \"8b505246-b351-447c-aa7d-711bde6225e3\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"
Sep 10 21:07:21.943574 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.943499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8b505246-b351-447c-aa7d-711bde6225e3-config-file\") pod \"limitador-limitador-84994f7ffc-mmj7x\" (UID: \"8b505246-b351-447c-aa7d-711bde6225e3\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"
Sep 10 21:07:21.944041 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.944024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8b505246-b351-447c-aa7d-711bde6225e3-config-file\") pod \"limitador-limitador-84994f7ffc-mmj7x\" (UID: \"8b505246-b351-447c-aa7d-711bde6225e3\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"
Sep 10 21:07:21.951939 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:21.951916    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5hb8\" (UniqueName: \"kubernetes.io/projected/8b505246-b351-447c-aa7d-711bde6225e3-kube-api-access-k5hb8\") pod \"limitador-limitador-84994f7ffc-mmj7x\" (UID: \"8b505246-b351-447c-aa7d-711bde6225e3\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"
Sep 10 21:07:22.057192 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:22.057171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"
Sep 10 21:07:22.235732 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:22.235697    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:22.321471 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:22.321408    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:22.328179 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:22.328084    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:07:22.330956 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:07:22.330934    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b505246_b351_447c_aa7d_711bde6225e3.slice/crio-67f83fc8949ae7c526dbae0445ac44a00a81ad1cf65c7e015de23dd8e36d559b WatchSource:0}: Error finding container 67f83fc8949ae7c526dbae0445ac44a00a81ad1cf65c7e015de23dd8e36d559b: Status 404 returned error can't find the container with id 67f83fc8949ae7c526dbae0445ac44a00a81ad1cf65c7e015de23dd8e36d559b
Sep 10 21:07:22.358109 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:22.358036    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-mmj7x" event={"ID":"8b505246-b351-447c-aa7d-711bde6225e3","Type":"ContainerStarted","Data":"67f83fc8949ae7c526dbae0445ac44a00a81ad1cf65c7e015de23dd8e36d559b"}
Sep 10 21:07:26.380369 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:26.380334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-mmj7x" event={"ID":"8b505246-b351-447c-aa7d-711bde6225e3","Type":"ContainerStarted","Data":"4e155438109c704c2bd29cfc5a13c34fb94e4d97c83dd60a367ea704185a221d"}
Sep 10 21:07:26.380781 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:26.380424    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"
Sep 10 21:07:26.398242 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:26.398196    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-mmj7x" podStartSLOduration=2.002223027 podStartE2EDuration="5.398179564s" totalImagesPullingTime="3.395956537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:07:26.39710893 +0000 UTC m=+540.076607781" watchObservedRunningTime="2026-09-10 21:07:26.398179564 +0000 UTC m=+540.077678413"
Sep 10 21:07:37.386563 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:37.386518    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"
Sep 10 21:07:37.581751 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:37.581720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:43.615598 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:43.615563    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:43.713586 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:43.713558    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:44.119402 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:44.119373    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:56.434889 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:56.434845    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:07:56.669383 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:07:56.669339    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:04.147460 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:04.147427    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:04.252978 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:04.252908    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:04.347421 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:04.347368    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:04.587640 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:04.587612    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:04.938473 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:04.938399    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:08.452666 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:08.452621    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:08.554510 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:08.554480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:41.114856 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:41.114818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:41.202564 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:41.202534    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:42.711495 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:42.711467    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:42.801445 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:42.801412    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:42.903445 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:42.903415    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:43.303206 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.303175    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:43.407202 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.407165    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:43.416669 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.416646    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7"]
Sep 10 21:08:43.419914 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.419899    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7"
Sep 10 21:08:43.423362 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.423340    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-6pvvs\""
Sep 10 21:08:43.423444 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.423388    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 21:08:43.431286 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.431260    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7"]
Sep 10 21:08:43.451074 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.448793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c795816d-ea80-46bf-9ed8-bace71464e54-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-q52r7\" (UID: \"c795816d-ea80-46bf-9ed8-bace71464e54\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7"
Sep 10 21:08:43.451074 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.448883    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhjtw\" (UniqueName: \"kubernetes.io/projected/c795816d-ea80-46bf-9ed8-bace71464e54-kube-api-access-rhjtw\") pod \"payload-pre-processing-59fbc47bf8-q52r7\" (UID: \"c795816d-ea80-46bf-9ed8-bace71464e54\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7"
Sep 10 21:08:43.549568 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.549542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c795816d-ea80-46bf-9ed8-bace71464e54-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-q52r7\" (UID: \"c795816d-ea80-46bf-9ed8-bace71464e54\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7"
Sep 10 21:08:43.549691 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.549594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhjtw\" (UniqueName: \"kubernetes.io/projected/c795816d-ea80-46bf-9ed8-bace71464e54-kube-api-access-rhjtw\") pod \"payload-pre-processing-59fbc47bf8-q52r7\" (UID: \"c795816d-ea80-46bf-9ed8-bace71464e54\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7"
Sep 10 21:08:43.550316 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.550295    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c795816d-ea80-46bf-9ed8-bace71464e54-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-q52r7\" (UID: \"c795816d-ea80-46bf-9ed8-bace71464e54\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7"
Sep 10 21:08:43.559727 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.559674    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:08:43.571366 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.571347    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhjtw\" (UniqueName: \"kubernetes.io/projected/c795816d-ea80-46bf-9ed8-bace71464e54-kube-api-access-rhjtw\") pod \"payload-pre-processing-59fbc47bf8-q52r7\" (UID: \"c795816d-ea80-46bf-9ed8-bace71464e54\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7"
Sep 10 21:08:43.730720 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:43.730697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7"
Sep 10 21:08:44.006814 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:44.006785    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7"]
Sep 10 21:08:44.008351 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:08:44.008326    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc795816d_ea80_46bf_9ed8_bace71464e54.slice/crio-6bfe44376cb56f8fdc347370e6b3a99eb7278a47fbf82cc1a94239fecea48edf WatchSource:0}: Error finding container 6bfe44376cb56f8fdc347370e6b3a99eb7278a47fbf82cc1a94239fecea48edf: Status 404 returned error can't find the container with id 6bfe44376cb56f8fdc347370e6b3a99eb7278a47fbf82cc1a94239fecea48edf
Sep 10 21:08:44.010248 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:44.010227    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:08:44.768510 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:44.768470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7" event={"ID":"c795816d-ea80-46bf-9ed8-bace71464e54","Type":"ContainerStarted","Data":"6bfe44376cb56f8fdc347370e6b3a99eb7278a47fbf82cc1a94239fecea48edf"}
Sep 10 21:08:46.778243 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:46.778210    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7" event={"ID":"c795816d-ea80-46bf-9ed8-bace71464e54","Type":"ContainerStarted","Data":"fbae9709b99f94e7d0084f2949baf6921934a692aa11abea42dd3b56dcd6e17b"}
Sep 10 21:08:46.778607 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:46.778277    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7"
Sep 10 21:08:46.797531 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:46.797490    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7" podStartSLOduration=2.086926138 podStartE2EDuration="3.797475523s" totalImagesPullingTime="1.710549385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:08:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:08:46.795826814 +0000 UTC m=+620.475325675" watchObservedRunningTime="2026-09-10 21:08:46.797475523 +0000 UTC m=+620.476974373"
Sep 10 21:08:52.450530 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:52.450442    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:52.537266 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:52.537236    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:08:57.784383 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:08:57.784351    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q52r7"
Sep 10 21:09:05.586478 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:05.586443    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:09:06.042783 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:06.042740    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:09:18.617605 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:18.617559    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:09:19.240602 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:19.240568    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:09:24.863895 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.863864    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"]
Sep 10 21:09:24.867508 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.867486    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:24.871191 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.871170    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 21:09:24.871370 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.871176    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kxt89\""
Sep 10 21:09:24.871495 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.871182    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 21:09:24.871495 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.871274    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 21:09:24.876781 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.876760    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"]
Sep 10 21:09:24.893449 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.893428    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:09:24.972278 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.972256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr9qc\" (UniqueName: \"kubernetes.io/projected/a957e972-8424-4353-960a-1fc163fdfb2c-kube-api-access-kr9qc\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:24.972392 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.972291    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a957e972-8424-4353-960a-1fc163fdfb2c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:24.972392 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.972331    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a957e972-8424-4353-960a-1fc163fdfb2c-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:24.972466 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.972392    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a957e972-8424-4353-960a-1fc163fdfb2c-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:24.972466 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.972417    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a957e972-8424-4353-960a-1fc163fdfb2c-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:24.972529 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:24.972460    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a957e972-8424-4353-960a-1fc163fdfb2c-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.072936 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.072911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a957e972-8424-4353-960a-1fc163fdfb2c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.073040 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.072960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a957e972-8424-4353-960a-1fc163fdfb2c-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.073040 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.072995    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a957e972-8424-4353-960a-1fc163fdfb2c-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.073040 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.073016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a957e972-8424-4353-960a-1fc163fdfb2c-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.073040 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.073036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a957e972-8424-4353-960a-1fc163fdfb2c-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.073334 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.073051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr9qc\" (UniqueName: \"kubernetes.io/projected/a957e972-8424-4353-960a-1fc163fdfb2c-kube-api-access-kr9qc\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.073391 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.073350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a957e972-8424-4353-960a-1fc163fdfb2c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.073448 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.073424    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a957e972-8424-4353-960a-1fc163fdfb2c-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.073501 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.073475    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a957e972-8424-4353-960a-1fc163fdfb2c-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.075325 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.075306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a957e972-8424-4353-960a-1fc163fdfb2c-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.075586 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.075566    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a957e972-8424-4353-960a-1fc163fdfb2c-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.083461 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.083438    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr9qc\" (UniqueName: \"kubernetes.io/projected/a957e972-8424-4353-960a-1fc163fdfb2c-kube-api-access-kr9qc\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b\" (UID: \"a957e972-8424-4353-960a-1fc163fdfb2c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.178717 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.178657    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:25.450337 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.450310    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"]
Sep 10 21:09:25.452716 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:09:25.452690    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda957e972_8424_4353_960a_1fc163fdfb2c.slice/crio-5cd6176c7b7c93fd5714645d5e796fa99375628e09c69ef2772f83047c590997 WatchSource:0}: Error finding container 5cd6176c7b7c93fd5714645d5e796fa99375628e09c69ef2772f83047c590997: Status 404 returned error can't find the container with id 5cd6176c7b7c93fd5714645d5e796fa99375628e09c69ef2772f83047c590997
Sep 10 21:09:25.838626 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.838595    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:09:25.960814 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:25.960770    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b" event={"ID":"a957e972-8424-4353-960a-1fc163fdfb2c","Type":"ContainerStarted","Data":"5cd6176c7b7c93fd5714645d5e796fa99375628e09c69ef2772f83047c590997"}
Sep 10 21:09:26.965793 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:26.965759    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b" event={"ID":"a957e972-8424-4353-960a-1fc163fdfb2c","Type":"ContainerStarted","Data":"b7ea10b0193ca84c862649ab8d54543235c9f3e336907f0c65233d5ea9405f7f"}
Sep 10 21:09:26.986648 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:26.986605    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b" podStartSLOduration=2.022532886 podStartE2EDuration="2.986589717s" totalImagesPullingTime="964.056831ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:09:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:09:26.9854685 +0000 UTC m=+660.664967350" watchObservedRunningTime="2026-09-10 21:09:26.986589717 +0000 UTC m=+660.666088566"
Sep 10 21:09:35.179495 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:35.179462    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:35.179867 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:35.179522    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:35.191852 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:35.191823    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:35.559073 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:35.559036    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:09:36.017825 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:36.017746    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b"
Sep 10 21:09:45.596865 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:45.596817    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:09:46.642299 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:46.642262    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:09:55.106297 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:55.106239    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:09:56.542519 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:09:56.542488    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:10:04.663459 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.663428    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"]
Sep 10 21:10:04.668880 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.668859    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.671522 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.671500    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 21:10:04.680094 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.678933    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"]
Sep 10 21:10:04.692549 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.692525    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:04.768694 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.768670    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/717b2af5-03e4-4fa2-9440-16be94a5c6fb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.768792 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.768697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/717b2af5-03e4-4fa2-9440-16be94a5c6fb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.768792 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.768728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/717b2af5-03e4-4fa2-9440-16be94a5c6fb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.768792 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.768769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/717b2af5-03e4-4fa2-9440-16be94a5c6fb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.768892 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.768832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhsxh\" (UniqueName: \"kubernetes.io/projected/717b2af5-03e4-4fa2-9440-16be94a5c6fb-kube-api-access-hhsxh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.768892 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.768852    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/717b2af5-03e4-4fa2-9440-16be94a5c6fb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.869534 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.869467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/717b2af5-03e4-4fa2-9440-16be94a5c6fb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.869534 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.869504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/717b2af5-03e4-4fa2-9440-16be94a5c6fb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.869698 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.869549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/717b2af5-03e4-4fa2-9440-16be94a5c6fb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.869698 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.869573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/717b2af5-03e4-4fa2-9440-16be94a5c6fb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.869698 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.869601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhsxh\" (UniqueName: \"kubernetes.io/projected/717b2af5-03e4-4fa2-9440-16be94a5c6fb-kube-api-access-hhsxh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.869698 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.869626    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/717b2af5-03e4-4fa2-9440-16be94a5c6fb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.869895 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.869837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/717b2af5-03e4-4fa2-9440-16be94a5c6fb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.869951 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.869911    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/717b2af5-03e4-4fa2-9440-16be94a5c6fb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.870010 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.869988    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/717b2af5-03e4-4fa2-9440-16be94a5c6fb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.871777 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.871750    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/717b2af5-03e4-4fa2-9440-16be94a5c6fb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.872143 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.872121    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/717b2af5-03e4-4fa2-9440-16be94a5c6fb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.879566 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.879545    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhsxh\" (UniqueName: \"kubernetes.io/projected/717b2af5-03e4-4fa2-9440-16be94a5c6fb-kube-api-access-hhsxh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54974\" (UID: \"717b2af5-03e4-4fa2-9440-16be94a5c6fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:04.984432 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:04.984408    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:05.256926 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:05.256901    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"]
Sep 10 21:10:05.258924 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:10:05.258895    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod717b2af5_03e4_4fa2_9440_16be94a5c6fb.slice/crio-d6320cdaf73b3d2db5115cdd7fa3c75afecb406ef36acf6a2ee6a38eb8d590e8 WatchSource:0}: Error finding container d6320cdaf73b3d2db5115cdd7fa3c75afecb406ef36acf6a2ee6a38eb8d590e8: Status 404 returned error can't find the container with id d6320cdaf73b3d2db5115cdd7fa3c75afecb406ef36acf6a2ee6a38eb8d590e8
Sep 10 21:10:06.139808 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:06.139771    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974" event={"ID":"717b2af5-03e4-4fa2-9440-16be94a5c6fb","Type":"ContainerStarted","Data":"85f9ad5b20b77140e19a110109c1b86335b9758bfffc7442f7790e36e4dfa642"}
Sep 10 21:10:06.140295 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:06.139827    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974" event={"ID":"717b2af5-03e4-4fa2-9440-16be94a5c6fb","Type":"ContainerStarted","Data":"d6320cdaf73b3d2db5115cdd7fa3c75afecb406ef36acf6a2ee6a38eb8d590e8"}
Sep 10 21:10:06.159028 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:06.158981    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974" podStartSLOduration=1.934611731 podStartE2EDuration="2.158963403s" totalImagesPullingTime="224.351672ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:06.15759308 +0000 UTC m=+699.837091929" watchObservedRunningTime="2026-09-10 21:10:06.158963403 +0000 UTC m=+699.838462255"
Sep 10 21:10:06.445471 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:06.445389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:10:12.599638 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:12.599606    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:14.636504 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:14.636471    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:10:14.984715 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:14.984627    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:14.984715 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:14.984691    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:14.997131 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:14.997100    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:15.194000 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:15.193971    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54974"
Sep 10 21:10:34.973838 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:34.973762    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:35.031994 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:35.031965    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:43.039493 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:43.039442    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:10:55.870272 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:10:55.870233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:09.087526 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:09.087487    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:09.188564 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:09.188525    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:32.793879 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:32.793842    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:32.889867 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:32.889829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:33.970740 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:33.970698    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:34.070785 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:34.070752    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:49.912581 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:49.912493    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:50.029562 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:50.029523    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:50.264254 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:50.264166    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:50.523836 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:50.523746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:51.426335 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:51.426296    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:51.528816 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:51.528773    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:51.924362 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:51.924324    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:53.165579 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:53.165549    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:53.265641 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:53.265604    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:53.744123 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:53.744087    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:11:54.374154 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:54.374121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:54.474835 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:54.474795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:54.591586 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:54.591552    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:54.952885 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:54.952836    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:55.054879 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.054842    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:55.056741 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.056714    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"]
Sep 10 21:11:55.060989 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.060961    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"
Sep 10 21:11:55.063555 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.063528    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-b22a78-dockercfg-vk967\""
Sep 10 21:11:55.063677 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.063585    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-b22a78\""
Sep 10 21:11:55.071011 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.070971    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"]
Sep 10 21:11:55.099840 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.099803    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"]
Sep 10 21:11:55.103374 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.103354    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"
Sep 10 21:11:55.113234 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.113206    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"]
Sep 10 21:11:55.134819 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.134788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzjrb\" (UniqueName: \"kubernetes.io/projected/e252a656-33bf-482c-aeeb-71b9b2022f48-kube-api-access-zzjrb\") pod \"payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj\" (UID: \"e252a656-33bf-482c-aeeb-71b9b2022f48\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"
Sep 10 21:11:55.134819 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.134826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e252a656-33bf-482c-aeeb-71b9b2022f48-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj\" (UID: \"e252a656-33bf-482c-aeeb-71b9b2022f48\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"
Sep 10 21:11:55.236028 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.235924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzjrb\" (UniqueName: \"kubernetes.io/projected/e252a656-33bf-482c-aeeb-71b9b2022f48-kube-api-access-zzjrb\") pod \"payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj\" (UID: \"e252a656-33bf-482c-aeeb-71b9b2022f48\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"
Sep 10 21:11:55.236028 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.235971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e252a656-33bf-482c-aeeb-71b9b2022f48-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj\" (UID: \"e252a656-33bf-482c-aeeb-71b9b2022f48\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"
Sep 10 21:11:55.236281 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.236042    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ceaf786-2b54-45f1-9e05-171db5b1feb6-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv\" (UID: \"1ceaf786-2b54-45f1-9e05-171db5b1feb6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"
Sep 10 21:11:55.236281 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.236119    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nlsn\" (UniqueName: \"kubernetes.io/projected/1ceaf786-2b54-45f1-9e05-171db5b1feb6-kube-api-access-7nlsn\") pod \"payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv\" (UID: \"1ceaf786-2b54-45f1-9e05-171db5b1feb6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"
Sep 10 21:11:55.236722 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.236700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e252a656-33bf-482c-aeeb-71b9b2022f48-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj\" (UID: \"e252a656-33bf-482c-aeeb-71b9b2022f48\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"
Sep 10 21:11:55.245165 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.245129    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzjrb\" (UniqueName: \"kubernetes.io/projected/e252a656-33bf-482c-aeeb-71b9b2022f48-kube-api-access-zzjrb\") pod \"payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj\" (UID: \"e252a656-33bf-482c-aeeb-71b9b2022f48\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"
Sep 10 21:11:55.336995 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.336955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nlsn\" (UniqueName: \"kubernetes.io/projected/1ceaf786-2b54-45f1-9e05-171db5b1feb6-kube-api-access-7nlsn\") pod \"payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv\" (UID: \"1ceaf786-2b54-45f1-9e05-171db5b1feb6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"
Sep 10 21:11:55.337202 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.337104    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ceaf786-2b54-45f1-9e05-171db5b1feb6-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv\" (UID: \"1ceaf786-2b54-45f1-9e05-171db5b1feb6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"
Sep 10 21:11:55.337820 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.337797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ceaf786-2b54-45f1-9e05-171db5b1feb6-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv\" (UID: \"1ceaf786-2b54-45f1-9e05-171db5b1feb6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"
Sep 10 21:11:55.346340 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.346315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nlsn\" (UniqueName: \"kubernetes.io/projected/1ceaf786-2b54-45f1-9e05-171db5b1feb6-kube-api-access-7nlsn\") pod \"payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv\" (UID: \"1ceaf786-2b54-45f1-9e05-171db5b1feb6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"
Sep 10 21:11:55.374372 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.374316    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"
Sep 10 21:11:55.416815 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.416769    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"
Sep 10 21:11:55.652419 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.652384    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"]
Sep 10 21:11:55.656116 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:11:55.656089    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode252a656_33bf_482c_aeeb_71b9b2022f48.slice/crio-00030b2c7ce43768c4cb6a94458005f4c7c3e8a38255c08142086c8afd60fcce WatchSource:0}: Error finding container 00030b2c7ce43768c4cb6a94458005f4c7c3e8a38255c08142086c8afd60fcce: Status 404 returned error can't find the container with id 00030b2c7ce43768c4cb6a94458005f4c7c3e8a38255c08142086c8afd60fcce
Sep 10 21:11:55.665957 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.665923    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj" event={"ID":"e252a656-33bf-482c-aeeb-71b9b2022f48","Type":"ContainerStarted","Data":"00030b2c7ce43768c4cb6a94458005f4c7c3e8a38255c08142086c8afd60fcce"}
Sep 10 21:11:55.709154 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:55.709126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"]
Sep 10 21:11:55.711837 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:11:55.711809    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ceaf786_2b54_45f1_9e05_171db5b1feb6.slice/crio-74e9501b4b1139a4e7e6f42118817b302c88886038ed752afb0b2b1c0cb4be0c WatchSource:0}: Error finding container 74e9501b4b1139a4e7e6f42118817b302c88886038ed752afb0b2b1c0cb4be0c: Status 404 returned error can't find the container with id 74e9501b4b1139a4e7e6f42118817b302c88886038ed752afb0b2b1c0cb4be0c
Sep 10 21:11:56.672592 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:56.672551    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv" event={"ID":"1ceaf786-2b54-45f1-9e05-171db5b1feb6","Type":"ContainerStarted","Data":"f80dbd28d5a0a75f9255126e0519309d18547389fee5717b79b02d6db1a216fb"}
Sep 10 21:11:56.673073 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:56.672653    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv" event={"ID":"1ceaf786-2b54-45f1-9e05-171db5b1feb6","Type":"ContainerStarted","Data":"74e9501b4b1139a4e7e6f42118817b302c88886038ed752afb0b2b1c0cb4be0c"}
Sep 10 21:11:56.673073 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:56.672686    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"
Sep 10 21:11:56.674181 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:56.674156    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj" event={"ID":"e252a656-33bf-482c-aeeb-71b9b2022f48","Type":"ContainerStarted","Data":"e7d71109a4e1e7c94ff4dc2fbcb68410939fcb20d626f5ae059468f3cd59055d"}
Sep 10 21:11:56.674296 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:56.674283    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"
Sep 10 21:11:56.694173 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:56.694103    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv" podStartSLOduration=1.6940786289999998 podStartE2EDuration="1.694078629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:56.691957361 +0000 UTC m=+810.371456214" watchObservedRunningTime="2026-09-10 21:11:56.694078629 +0000 UTC m=+810.373577477"
Sep 10 21:11:56.709681 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:56.709610    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj" podStartSLOduration=1.7095856299999999 podStartE2EDuration="1.70958563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:56.70728431 +0000 UTC m=+810.386783165" watchObservedRunningTime="2026-09-10 21:11:56.70958563 +0000 UTC m=+810.389084485"
Sep 10 21:11:57.996107 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:57.996070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:58.056491 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:58.056441    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:58.241950 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:58.241914    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:58.536105 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:58.536030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:58.636080 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:58.636023    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:58.761503 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:58.761470    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:58.882840 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:58.882796    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:11:59.131105 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:11:59.131025    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:01.010676 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:01.010625    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:01.590210 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:01.590177    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:01.681764 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:01.681715    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:02.755596 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:02.755562    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:02.847429 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:02.847393    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:03.186344 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.186314    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:03.280342 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.280308    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:03.323638 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.323596    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"]
Sep 10 21:12:03.327797 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.327767    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"
Sep 10 21:12:03.330196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.330168    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-8086f00e\""
Sep 10 21:12:03.339829 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.339799    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"]
Sep 10 21:12:03.355336 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.355301    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:03.362575 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.362543    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"]
Sep 10 21:12:03.366251 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.366227    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"
Sep 10 21:12:03.368753 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.368723    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-8086f00e-dockercfg-85vcz\""
Sep 10 21:12:03.375925 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.375901    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"]
Sep 10 21:12:03.417419 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.417388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cc68aea-9cb5-433c-8472-3b80171e95bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv\" (UID: \"6cc68aea-9cb5-433c-8472-3b80171e95bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"
Sep 10 21:12:03.417589 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.417427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwqbk\" (UniqueName: \"kubernetes.io/projected/6cc68aea-9cb5-433c-8472-3b80171e95bf-kube-api-access-gwqbk\") pod \"payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv\" (UID: \"6cc68aea-9cb5-433c-8472-3b80171e95bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"
Sep 10 21:12:03.518778 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.518676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c83aa30d-af1b-4fd6-804f-78abef1307ff-plugins-config-volume\") pod \"payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt\" (UID: \"c83aa30d-af1b-4fd6-804f-78abef1307ff\") " pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"
Sep 10 21:12:03.518947 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.518851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cc68aea-9cb5-433c-8472-3b80171e95bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv\" (UID: \"6cc68aea-9cb5-433c-8472-3b80171e95bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"
Sep 10 21:12:03.518947 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.518886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgdrj\" (UniqueName: \"kubernetes.io/projected/c83aa30d-af1b-4fd6-804f-78abef1307ff-kube-api-access-vgdrj\") pod \"payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt\" (UID: \"c83aa30d-af1b-4fd6-804f-78abef1307ff\") " pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"
Sep 10 21:12:03.518947 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.518916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwqbk\" (UniqueName: \"kubernetes.io/projected/6cc68aea-9cb5-433c-8472-3b80171e95bf-kube-api-access-gwqbk\") pod \"payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv\" (UID: \"6cc68aea-9cb5-433c-8472-3b80171e95bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"
Sep 10 21:12:03.519631 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.519604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cc68aea-9cb5-433c-8472-3b80171e95bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv\" (UID: \"6cc68aea-9cb5-433c-8472-3b80171e95bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"
Sep 10 21:12:03.527450 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.527420    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwqbk\" (UniqueName: \"kubernetes.io/projected/6cc68aea-9cb5-433c-8472-3b80171e95bf-kube-api-access-gwqbk\") pod \"payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv\" (UID: \"6cc68aea-9cb5-433c-8472-3b80171e95bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"
Sep 10 21:12:03.620439 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.620395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgdrj\" (UniqueName: \"kubernetes.io/projected/c83aa30d-af1b-4fd6-804f-78abef1307ff-kube-api-access-vgdrj\") pod \"payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt\" (UID: \"c83aa30d-af1b-4fd6-804f-78abef1307ff\") " pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"
Sep 10 21:12:03.620704 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.620455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c83aa30d-af1b-4fd6-804f-78abef1307ff-plugins-config-volume\") pod \"payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt\" (UID: \"c83aa30d-af1b-4fd6-804f-78abef1307ff\") " pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"
Sep 10 21:12:03.621202 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.621183    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c83aa30d-af1b-4fd6-804f-78abef1307ff-plugins-config-volume\") pod \"payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt\" (UID: \"c83aa30d-af1b-4fd6-804f-78abef1307ff\") " pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"
Sep 10 21:12:03.629289 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.629262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgdrj\" (UniqueName: \"kubernetes.io/projected/c83aa30d-af1b-4fd6-804f-78abef1307ff-kube-api-access-vgdrj\") pod \"payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt\" (UID: \"c83aa30d-af1b-4fd6-804f-78abef1307ff\") " pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"
Sep 10 21:12:03.640176 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.640148    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"
Sep 10 21:12:03.677875 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.677832    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"
Sep 10 21:12:03.918202 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.918171    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"]
Sep 10 21:12:03.920516 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:12:03.920490    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc68aea_9cb5_433c_8472_3b80171e95bf.slice/crio-9d01594a28bc1593c81c0d379ec3f102a1a0e0aa85af661f978fe98fd0f5e238 WatchSource:0}: Error finding container 9d01594a28bc1593c81c0d379ec3f102a1a0e0aa85af661f978fe98fd0f5e238: Status 404 returned error can't find the container with id 9d01594a28bc1593c81c0d379ec3f102a1a0e0aa85af661f978fe98fd0f5e238
Sep 10 21:12:03.971673 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:03.971621    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"]
Sep 10 21:12:03.973644 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:12:03.973617    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc83aa30d_af1b_4fd6_804f_78abef1307ff.slice/crio-ca4e8ad7bb243318c0e067c2f2dfb57ed82ff9b066d6812ef21c7d98f10b8b4c WatchSource:0}: Error finding container ca4e8ad7bb243318c0e067c2f2dfb57ed82ff9b066d6812ef21c7d98f10b8b4c: Status 404 returned error can't find the container with id ca4e8ad7bb243318c0e067c2f2dfb57ed82ff9b066d6812ef21c7d98f10b8b4c
Sep 10 21:12:04.395928 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:04.395888    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:04.461947 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:04.461911    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:04.724026 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:04.723924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt" event={"ID":"c83aa30d-af1b-4fd6-804f-78abef1307ff","Type":"ContainerStarted","Data":"b2f61c149a421480462d2d5cdc8ddf089b5c00a1741ad4e7127db1969701144d"}
Sep 10 21:12:04.724026 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:04.723999    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"
Sep 10 21:12:04.724026 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:04.724023    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt" event={"ID":"c83aa30d-af1b-4fd6-804f-78abef1307ff","Type":"ContainerStarted","Data":"ca4e8ad7bb243318c0e067c2f2dfb57ed82ff9b066d6812ef21c7d98f10b8b4c"}
Sep 10 21:12:04.725822 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:04.725789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv" event={"ID":"6cc68aea-9cb5-433c-8472-3b80171e95bf","Type":"ContainerStarted","Data":"7043f250e414847a0b2e1e15f2445da4b6120a7d9e167ee7ee0e4ccecb837eb9"}
Sep 10 21:12:04.725981 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:04.725838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv" event={"ID":"6cc68aea-9cb5-433c-8472-3b80171e95bf","Type":"ContainerStarted","Data":"9d01594a28bc1593c81c0d379ec3f102a1a0e0aa85af661f978fe98fd0f5e238"}
Sep 10 21:12:04.725981 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:04.725860    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"
Sep 10 21:12:04.744692 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:04.744625    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt" podStartSLOduration=1.74460112 podStartE2EDuration="1.74460112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:04.741732642 +0000 UTC m=+818.421231494" watchObservedRunningTime="2026-09-10 21:12:04.74460112 +0000 UTC m=+818.424099971"
Sep 10 21:12:04.760837 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:04.760774    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv" podStartSLOduration=1.76074974 podStartE2EDuration="1.76074974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:04.758918682 +0000 UTC m=+818.438417586" watchObservedRunningTime="2026-09-10 21:12:04.76074974 +0000 UTC m=+818.440248593"
Sep 10 21:12:05.252156 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:05.247227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:12:05.543087 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:05.543042    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:05.651484 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:05.651451    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:05.835181 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:05.835051    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.079657 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.079581    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.179827 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.179740    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:06.207625 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.207588    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"]
Sep 10 21:12:06.211379 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.211360    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"
Sep 10 21:12:06.213857 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.213827    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-5e2fa5\""
Sep 10 21:12:06.222169 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.222138    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"]
Sep 10 21:12:06.345750 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.345700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b41f6c23-c31f-4cd7-8cd0-608c61ecbd60-plugins-config-volume\") pod \"payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz\" (UID: \"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60\") " pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"
Sep 10 21:12:06.346439 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.345839    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9zgw\" (UniqueName: \"kubernetes.io/projected/b41f6c23-c31f-4cd7-8cd0-608c61ecbd60-kube-api-access-v9zgw\") pod \"payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz\" (UID: \"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60\") " pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"
Sep 10 21:12:06.446965 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.446867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9zgw\" (UniqueName: \"kubernetes.io/projected/b41f6c23-c31f-4cd7-8cd0-608c61ecbd60-kube-api-access-v9zgw\") pod \"payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz\" (UID: \"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60\") " pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"
Sep 10 21:12:06.446965 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.446919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b41f6c23-c31f-4cd7-8cd0-608c61ecbd60-plugins-config-volume\") pod \"payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz\" (UID: \"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60\") " pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"
Sep 10 21:12:06.447664 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.447642    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b41f6c23-c31f-4cd7-8cd0-608c61ecbd60-plugins-config-volume\") pod \"payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz\" (UID: \"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60\") " pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"
Sep 10 21:12:06.456004 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.455978    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9zgw\" (UniqueName: \"kubernetes.io/projected/b41f6c23-c31f-4cd7-8cd0-608c61ecbd60-kube-api-access-v9zgw\") pod \"payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz\" (UID: \"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60\") " pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"
Sep 10 21:12:06.524206 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.524154    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"
Sep 10 21:12:06.801292 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:06.801266    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"]
Sep 10 21:12:06.803967 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:12:06.803938    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41f6c23_c31f_4cd7_8cd0_608c61ecbd60.slice/crio-965d0f0912db236267c24fac2695ae21f37fa4287e2e3d04c10666bd1097fefc WatchSource:0}: Error finding container 965d0f0912db236267c24fac2695ae21f37fa4287e2e3d04c10666bd1097fefc: Status 404 returned error can't find the container with id 965d0f0912db236267c24fac2695ae21f37fa4287e2e3d04c10666bd1097fefc
Sep 10 21:12:07.681626 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:07.681593    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"
Sep 10 21:12:07.682008 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:07.681675    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"
Sep 10 21:12:07.743926 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:07.743890    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz" event={"ID":"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60","Type":"ContainerStarted","Data":"a77a8b02cd4ac804b1685885d0f46060ddf34fe7ee1d84df83786d779bb3b189"}
Sep 10 21:12:07.744123 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:07.743946    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz" event={"ID":"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60","Type":"ContainerStarted","Data":"965d0f0912db236267c24fac2695ae21f37fa4287e2e3d04c10666bd1097fefc"}
Sep 10 21:12:07.744123 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:07.744024    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"
Sep 10 21:12:07.763379 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:07.763329    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz" podStartSLOduration=1.7633061190000001 podStartE2EDuration="1.763306119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:07.761303519 +0000 UTC m=+821.440802398" watchObservedRunningTime="2026-09-10 21:12:07.763306119 +0000 UTC m=+821.442804972"
Sep 10 21:12:08.957287 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:08.957251    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:09.057018 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:09.056984    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:09.158735 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:09.158695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:09.257317 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:09.257216    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:09.357474 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:09.357434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:12.090048 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:12.090010    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:12.284141 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:12.284105    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:13.833382 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:13.833346    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:15.313195 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:15.313155    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:15.422316 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:15.422279    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:15.513870 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:15.513834    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:15.668325 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:15.668244    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:15.733861 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:15.733831    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"
Sep 10 21:12:15.734032 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:15.733913    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"
Sep 10 21:12:18.156383 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:18.156351    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:18.498976 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:18.498879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:18.751512 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:18.751425    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"
Sep 10 21:12:18.802686 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:18.802653    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:18.902036 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:18.901997    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:21.347357 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:21.347321    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:12:22.092839 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:22.092804    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:22.193093 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:22.193036    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:22.313439 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:22.313400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:22.399429 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:22.399341    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:23.541306 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:23.541266    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:23.653532 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:23.653492    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:23.824085 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:23.823989    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:23.998742 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:23.998707    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:24.095777 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.095682    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:24.103804 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.103766    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"]
Sep 10 21:12:24.107575 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.107550    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"
Sep 10 21:12:24.110159 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.110133    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-414d5f-dockercfg-k8qkc\""
Sep 10 21:12:24.110296 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.110138    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-414d5f\""
Sep 10 21:12:24.118839 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.118808    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"]
Sep 10 21:12:24.125657 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.125631    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"]
Sep 10 21:12:24.129513 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.129490    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"
Sep 10 21:12:24.141293 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.141230    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"]
Sep 10 21:12:24.196799 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.196754    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbrm9\" (UniqueName: \"kubernetes.io/projected/77a14455-f664-4f69-b08b-63efb0aa8898-kube-api-access-wbrm9\") pod \"payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5\" (UID: \"77a14455-f664-4f69-b08b-63efb0aa8898\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"
Sep 10 21:12:24.197010 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.196847    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6j2h\" (UniqueName: \"kubernetes.io/projected/659e7b52-25bc-4f02-850b-cda467acfc01-kube-api-access-n6j2h\") pod \"payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7\" (UID: \"659e7b52-25bc-4f02-850b-cda467acfc01\") " pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"
Sep 10 21:12:24.197010 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.196878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77a14455-f664-4f69-b08b-63efb0aa8898-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5\" (UID: \"77a14455-f664-4f69-b08b-63efb0aa8898\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"
Sep 10 21:12:24.197010 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.196911    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/659e7b52-25bc-4f02-850b-cda467acfc01-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7\" (UID: \"659e7b52-25bc-4f02-850b-cda467acfc01\") " pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"
Sep 10 21:12:24.298224 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.298180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrm9\" (UniqueName: \"kubernetes.io/projected/77a14455-f664-4f69-b08b-63efb0aa8898-kube-api-access-wbrm9\") pod \"payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5\" (UID: \"77a14455-f664-4f69-b08b-63efb0aa8898\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"
Sep 10 21:12:24.298451 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.298254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6j2h\" (UniqueName: \"kubernetes.io/projected/659e7b52-25bc-4f02-850b-cda467acfc01-kube-api-access-n6j2h\") pod \"payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7\" (UID: \"659e7b52-25bc-4f02-850b-cda467acfc01\") " pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"
Sep 10 21:12:24.298451 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.298276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77a14455-f664-4f69-b08b-63efb0aa8898-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5\" (UID: \"77a14455-f664-4f69-b08b-63efb0aa8898\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"
Sep 10 21:12:24.298451 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.298300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/659e7b52-25bc-4f02-850b-cda467acfc01-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7\" (UID: \"659e7b52-25bc-4f02-850b-cda467acfc01\") " pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"
Sep 10 21:12:24.299020 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.299001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/659e7b52-25bc-4f02-850b-cda467acfc01-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7\" (UID: \"659e7b52-25bc-4f02-850b-cda467acfc01\") " pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"
Sep 10 21:12:24.299098 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.299006    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77a14455-f664-4f69-b08b-63efb0aa8898-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5\" (UID: \"77a14455-f664-4f69-b08b-63efb0aa8898\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"
Sep 10 21:12:24.308492 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.308458    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6j2h\" (UniqueName: \"kubernetes.io/projected/659e7b52-25bc-4f02-850b-cda467acfc01-kube-api-access-n6j2h\") pod \"payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7\" (UID: \"659e7b52-25bc-4f02-850b-cda467acfc01\") " pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"
Sep 10 21:12:24.308645 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.308563    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbrm9\" (UniqueName: \"kubernetes.io/projected/77a14455-f664-4f69-b08b-63efb0aa8898-kube-api-access-wbrm9\") pod \"payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5\" (UID: \"77a14455-f664-4f69-b08b-63efb0aa8898\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"
Sep 10 21:12:24.366924 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.366824    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:24.421173 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.421131    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"
Sep 10 21:12:24.440984 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.440954    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"
Sep 10 21:12:24.702005 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.701976    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"]
Sep 10 21:12:24.703634 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:12:24.703607    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a14455_f664_4f69_b08b_63efb0aa8898.slice/crio-8242e0a43bfd2b5c4db9edfe2bbbe87ee36ddf565bd5353f38790847ce110699 WatchSource:0}: Error finding container 8242e0a43bfd2b5c4db9edfe2bbbe87ee36ddf565bd5353f38790847ce110699: Status 404 returned error can't find the container with id 8242e0a43bfd2b5c4db9edfe2bbbe87ee36ddf565bd5353f38790847ce110699
Sep 10 21:12:24.727641 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.727618    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"]
Sep 10 21:12:24.738266 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:12:24.738238    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod659e7b52_25bc_4f02_850b_cda467acfc01.slice/crio-81929d0f481acd91a1d782c5e1fc9e9d5d9d4e373edca3701a2f7607cd8ac20d WatchSource:0}: Error finding container 81929d0f481acd91a1d782c5e1fc9e9d5d9d4e373edca3701a2f7607cd8ac20d: Status 404 returned error can't find the container with id 81929d0f481acd91a1d782c5e1fc9e9d5d9d4e373edca3701a2f7607cd8ac20d
Sep 10 21:12:24.836177 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.836123    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5" event={"ID":"77a14455-f664-4f69-b08b-63efb0aa8898","Type":"ContainerStarted","Data":"bfc56ab4d9fbe3ba979ae4ced1fafa54233f21c4cf362968b220c9c1278d9562"}
Sep 10 21:12:24.836380 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.836188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5" event={"ID":"77a14455-f664-4f69-b08b-63efb0aa8898","Type":"ContainerStarted","Data":"8242e0a43bfd2b5c4db9edfe2bbbe87ee36ddf565bd5353f38790847ce110699"}
Sep 10 21:12:24.836380 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.836289    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"
Sep 10 21:12:24.838147 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.838113    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7" event={"ID":"659e7b52-25bc-4f02-850b-cda467acfc01","Type":"ContainerStarted","Data":"3059028c6b7fcbd9c35cf43c7462481c1141b16aefdb923503d9822bf62e9c67"}
Sep 10 21:12:24.838291 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.838179    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"
Sep 10 21:12:24.838291 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.838204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7" event={"ID":"659e7b52-25bc-4f02-850b-cda467acfc01","Type":"ContainerStarted","Data":"81929d0f481acd91a1d782c5e1fc9e9d5d9d4e373edca3701a2f7607cd8ac20d"}
Sep 10 21:12:24.855136 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.855079    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5" podStartSLOduration=0.855044245 podStartE2EDuration="855.044245ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:24.852966588 +0000 UTC m=+838.532465450" watchObservedRunningTime="2026-09-10 21:12:24.855044245 +0000 UTC m=+838.534543095"
Sep 10 21:12:24.870239 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:24.870177    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7" podStartSLOduration=0.87015715 podStartE2EDuration="870.15715ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:24.868445517 +0000 UTC m=+838.547944378" watchObservedRunningTime="2026-09-10 21:12:24.87015715 +0000 UTC m=+838.549656001"
Sep 10 21:12:25.098447 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:25.098414    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:25.950615 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:25.950575    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:26.688484 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:26.688452    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:27.406359 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:27.406319    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:28.167847 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:28.167809    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:29.398817 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:29.398777    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:35.846001 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:35.845970    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"
Sep 10 21:12:35.846396 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:35.846053    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"
Sep 10 21:12:39.328317 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:39.328275    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:12:43.161128 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:43.161086    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:43.722140 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:43.722049    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:43.752531 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:43.752494    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"]
Sep 10 21:12:43.752811 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:43.752785    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt" podUID="c83aa30d-af1b-4fd6-804f-78abef1307ff" containerName="payload-processing" containerID="cri-o://b2f61c149a421480462d2d5cdc8ddf089b5c00a1741ad4e7127db1969701144d" gracePeriod=30
Sep 10 21:12:43.760047 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:43.759992    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"]
Sep 10 21:12:43.760308 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:43.760257    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv" podUID="6cc68aea-9cb5-433c-8472-3b80171e95bf" containerName="payload-pre-processing" containerID="cri-o://7043f250e414847a0b2e1e15f2445da4b6120a7d9e167ee7ee0e4ccecb837eb9" gracePeriod=30
Sep 10 21:12:43.797829 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:43.797795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:43.855809 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:43.855771    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:43.912845 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:43.912807    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:43.946321 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:43.946290    2580 generic.go:356] "Generic (PLEG): container finished" podID="6cc68aea-9cb5-433c-8472-3b80171e95bf" containerID="7043f250e414847a0b2e1e15f2445da4b6120a7d9e167ee7ee0e4ccecb837eb9" exitCode=0
Sep 10 21:12:43.946520 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:43.946357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv" event={"ID":"6cc68aea-9cb5-433c-8472-3b80171e95bf","Type":"ContainerDied","Data":"7043f250e414847a0b2e1e15f2445da4b6120a7d9e167ee7ee0e4ccecb837eb9"}
Sep 10 21:12:43.948173 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:43.948145    2580 generic.go:356] "Generic (PLEG): container finished" podID="c83aa30d-af1b-4fd6-804f-78abef1307ff" containerID="b2f61c149a421480462d2d5cdc8ddf089b5c00a1741ad4e7127db1969701144d" exitCode=0
Sep 10 21:12:43.948297 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:43.948206    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt" event={"ID":"c83aa30d-af1b-4fd6-804f-78abef1307ff","Type":"ContainerDied","Data":"b2f61c149a421480462d2d5cdc8ddf089b5c00a1741ad4e7127db1969701144d"}
Sep 10 21:12:44.054720 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.054668    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:44.060696 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.060672    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"
Sep 10 21:12:44.064187 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.064162    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"
Sep 10 21:12:44.199392 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.199351    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cc68aea-9cb5-433c-8472-3b80171e95bf-plugins-config-volume\") pod \"6cc68aea-9cb5-433c-8472-3b80171e95bf\" (UID: \"6cc68aea-9cb5-433c-8472-3b80171e95bf\") "
Sep 10 21:12:44.199392 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.199395    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwqbk\" (UniqueName: \"kubernetes.io/projected/6cc68aea-9cb5-433c-8472-3b80171e95bf-kube-api-access-gwqbk\") pod \"6cc68aea-9cb5-433c-8472-3b80171e95bf\" (UID: \"6cc68aea-9cb5-433c-8472-3b80171e95bf\") "
Sep 10 21:12:44.199892 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.199531    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c83aa30d-af1b-4fd6-804f-78abef1307ff-plugins-config-volume\") pod \"c83aa30d-af1b-4fd6-804f-78abef1307ff\" (UID: \"c83aa30d-af1b-4fd6-804f-78abef1307ff\") "
Sep 10 21:12:44.199892 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.199551    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgdrj\" (UniqueName: \"kubernetes.io/projected/c83aa30d-af1b-4fd6-804f-78abef1307ff-kube-api-access-vgdrj\") pod \"c83aa30d-af1b-4fd6-804f-78abef1307ff\" (UID: \"c83aa30d-af1b-4fd6-804f-78abef1307ff\") "
Sep 10 21:12:44.199892 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.199858    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cc68aea-9cb5-433c-8472-3b80171e95bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6cc68aea-9cb5-433c-8472-3b80171e95bf" (UID: "6cc68aea-9cb5-433c-8472-3b80171e95bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:44.200038 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.199928    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c83aa30d-af1b-4fd6-804f-78abef1307ff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c83aa30d-af1b-4fd6-804f-78abef1307ff" (UID: "c83aa30d-af1b-4fd6-804f-78abef1307ff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:44.201863 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.201833    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c83aa30d-af1b-4fd6-804f-78abef1307ff-kube-api-access-vgdrj" (OuterVolumeSpecName: "kube-api-access-vgdrj") pod "c83aa30d-af1b-4fd6-804f-78abef1307ff" (UID: "c83aa30d-af1b-4fd6-804f-78abef1307ff"). InnerVolumeSpecName "kube-api-access-vgdrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:44.201949 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.201840    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cc68aea-9cb5-433c-8472-3b80171e95bf-kube-api-access-gwqbk" (OuterVolumeSpecName: "kube-api-access-gwqbk") pod "6cc68aea-9cb5-433c-8472-3b80171e95bf" (UID: "6cc68aea-9cb5-433c-8472-3b80171e95bf"). InnerVolumeSpecName "kube-api-access-gwqbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:44.301026 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.300987    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c83aa30d-af1b-4fd6-804f-78abef1307ff-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:12:44.301026 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.301022    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgdrj\" (UniqueName: \"kubernetes.io/projected/c83aa30d-af1b-4fd6-804f-78abef1307ff-kube-api-access-vgdrj\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:12:44.301026 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.301031    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cc68aea-9cb5-433c-8472-3b80171e95bf-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:12:44.301310 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.301040    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwqbk\" (UniqueName: \"kubernetes.io/projected/6cc68aea-9cb5-433c-8472-3b80171e95bf-kube-api-access-gwqbk\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:12:44.371715 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.371683    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:44.589189 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.588978    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:44.685768 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.685733    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:44.846958 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.846863    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:44.955879 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.955838    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"
Sep 10 21:12:44.957403 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.957379    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"
Sep 10 21:12:44.958117 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.958089    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv" event={"ID":"6cc68aea-9cb5-433c-8472-3b80171e95bf","Type":"ContainerDied","Data":"9d01594a28bc1593c81c0d379ec3f102a1a0e0aa85af661f978fe98fd0f5e238"}
Sep 10 21:12:44.958230 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.958143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt" event={"ID":"c83aa30d-af1b-4fd6-804f-78abef1307ff","Type":"ContainerDied","Data":"ca4e8ad7bb243318c0e067c2f2dfb57ed82ff9b066d6812ef21c7d98f10b8b4c"}
Sep 10 21:12:44.958230 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.958182    2580 scope.go:117] "RemoveContainer" containerID="7043f250e414847a0b2e1e15f2445da4b6120a7d9e167ee7ee0e4ccecb837eb9"
Sep 10 21:12:44.968269 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.968243    2580 scope.go:117] "RemoveContainer" containerID="b2f61c149a421480462d2d5cdc8ddf089b5c00a1741ad4e7127db1969701144d"
Sep 10 21:12:44.989643 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.989607    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"]
Sep 10 21:12:44.996247 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:44.996215    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8086f00e-5cdd8b859c-gfxgt"]
Sep 10 21:12:45.007931 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:45.007889    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"]
Sep 10 21:12:45.013810 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:45.013780    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8086f00e-7cb4bcfbbb-pzjhv"]
Sep 10 21:12:45.145905 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:45.145815    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:45.365505 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:45.365471    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:46.957146 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:46.957112    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cc68aea-9cb5-433c-8472-3b80171e95bf" path="/var/lib/kubelet/pods/6cc68aea-9cb5-433c-8472-3b80171e95bf/volumes"
Sep 10 21:12:46.957534 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:46.957483    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c83aa30d-af1b-4fd6-804f-78abef1307ff" path="/var/lib/kubelet/pods/c83aa30d-af1b-4fd6-804f-78abef1307ff/volumes"
Sep 10 21:12:47.333600 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.333561    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:47.554837 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.554801    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"]
Sep 10 21:12:47.555497 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.555474    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cc68aea-9cb5-433c-8472-3b80171e95bf" containerName="payload-pre-processing"
Sep 10 21:12:47.555497 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.555498    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cc68aea-9cb5-433c-8472-3b80171e95bf" containerName="payload-pre-processing"
Sep 10 21:12:47.555678 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.555522    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c83aa30d-af1b-4fd6-804f-78abef1307ff" containerName="payload-processing"
Sep 10 21:12:47.555678 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.555531    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c83aa30d-af1b-4fd6-804f-78abef1307ff" containerName="payload-processing"
Sep 10 21:12:47.555678 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.555631    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c83aa30d-af1b-4fd6-804f-78abef1307ff" containerName="payload-processing"
Sep 10 21:12:47.555678 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.555644    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cc68aea-9cb5-433c-8472-3b80171e95bf" containerName="payload-pre-processing"
Sep 10 21:12:47.560648 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.560622    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.563339 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.563312    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-1c65017b-kserve-self-signed-certs\""
Sep 10 21:12:47.568547 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.568518    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"]
Sep 10 21:12:47.582609 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.582557    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:47.635993 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.635899    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-tmp-dir\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.635993 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.635946    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-home\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.636228 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.636067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-dshm\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.636228 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.636122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-model-cache\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.636228 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.636191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f234e34-2574-4369-9cd6-e02d3c5da348-tls-certs\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.636228 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.636225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdfsw\" (UniqueName: \"kubernetes.io/projected/6f234e34-2574-4369-9cd6-e02d3c5da348-kube-api-access-sdfsw\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.736714 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.736669    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-model-cache\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.736912 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.736732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f234e34-2574-4369-9cd6-e02d3c5da348-tls-certs\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.736912 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.736759    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdfsw\" (UniqueName: \"kubernetes.io/projected/6f234e34-2574-4369-9cd6-e02d3c5da348-kube-api-access-sdfsw\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.736912 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.736787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-tmp-dir\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.736912 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.736811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-home\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.736912 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.736856    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-dshm\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.737217 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.737196    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-model-cache\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.737263 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.737215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-tmp-dir\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.737334 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.737317    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-home\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.739094 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.739052    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-dshm\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.739394 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.739375    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f234e34-2574-4369-9cd6-e02d3c5da348-tls-certs\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.746374 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.746345    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdfsw\" (UniqueName: \"kubernetes.io/projected/6f234e34-2574-4369-9cd6-e02d3c5da348-kube-api-access-sdfsw\") pod \"e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") " pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.872141 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.872090    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:47.985070 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:47.985018    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:48.085789 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.085753    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:48.146612 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.146583    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"]
Sep 10 21:12:48.148511 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:12:48.148485    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f234e34_2574_4369_9cd6_e02d3c5da348.slice/crio-4a135deb64f207942630b21233994e4fb127e864858cf43c28cbd42eefb5644a WatchSource:0}: Error finding container 4a135deb64f207942630b21233994e4fb127e864858cf43c28cbd42eefb5644a: Status 404 returned error can't find the container with id 4a135deb64f207942630b21233994e4fb127e864858cf43c28cbd42eefb5644a
Sep 10 21:12:48.283094 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.283041    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:48.365243 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.365208    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:48.740934 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.740906    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:48.847040 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.847004    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:48.885276 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.885243    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"]
Sep 10 21:12:48.889120 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.889097    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"
Sep 10 21:12:48.891563 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.891534    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-065d3bbf-dockercfg-4wx7n\""
Sep 10 21:12:48.891563 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.891534    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-065d3bbf\""
Sep 10 21:12:48.899365 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.899337    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"]
Sep 10 21:12:48.943224 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.943188    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:12:48.988993 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.988947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz" event={"ID":"6f234e34-2574-4369-9cd6-e02d3c5da348","Type":"ContainerStarted","Data":"d238d7616c8a00b5e7e3eca6a75a31b0879596d6182e9c63e3783cf2a1b8c892"}
Sep 10 21:12:48.989654 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:48.989007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz" event={"ID":"6f234e34-2574-4369-9cd6-e02d3c5da348","Type":"ContainerStarted","Data":"4a135deb64f207942630b21233994e4fb127e864858cf43c28cbd42eefb5644a"}
Sep 10 21:12:49.006998 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:49.006883    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz" podStartSLOduration=2.006865587 podStartE2EDuration="2.006865587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:49.005840605 +0000 UTC m=+862.685339479" watchObservedRunningTime="2026-09-10 21:12:49.006865587 +0000 UTC m=+862.686364437"
Sep 10 21:12:49.050719 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:49.050675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/257e5f98-8d38-4382-9b1a-6646b033c0a2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc\" (UID: \"257e5f98-8d38-4382-9b1a-6646b033c0a2\") " pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"
Sep 10 21:12:49.050903 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:49.050856    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9nf7\" (UniqueName: \"kubernetes.io/projected/257e5f98-8d38-4382-9b1a-6646b033c0a2-kube-api-access-g9nf7\") pod \"payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc\" (UID: \"257e5f98-8d38-4382-9b1a-6646b033c0a2\") " pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"
Sep 10 21:12:49.152186 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:49.152143    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9nf7\" (UniqueName: \"kubernetes.io/projected/257e5f98-8d38-4382-9b1a-6646b033c0a2-kube-api-access-g9nf7\") pod \"payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc\" (UID: \"257e5f98-8d38-4382-9b1a-6646b033c0a2\") " pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"
Sep 10 21:12:49.152367 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:49.152235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/257e5f98-8d38-4382-9b1a-6646b033c0a2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc\" (UID: \"257e5f98-8d38-4382-9b1a-6646b033c0a2\") " pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"
Sep 10 21:12:49.153032 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:49.153006    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/257e5f98-8d38-4382-9b1a-6646b033c0a2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc\" (UID: \"257e5f98-8d38-4382-9b1a-6646b033c0a2\") " pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"
Sep 10 21:12:49.160884 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:49.160843    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9nf7\" (UniqueName: \"kubernetes.io/projected/257e5f98-8d38-4382-9b1a-6646b033c0a2-kube-api-access-g9nf7\") pod \"payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc\" (UID: \"257e5f98-8d38-4382-9b1a-6646b033c0a2\") " pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"
Sep 10 21:12:49.202715 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:49.202676    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"
Sep 10 21:12:49.479947 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:49.479909    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"]
Sep 10 21:12:49.483009 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:12:49.482978    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod257e5f98_8d38_4382_9b1a_6646b033c0a2.slice/crio-1ecd96d51bf49bb7a4e1b000bfdb56a189f71a5d220db560e0eb6219f071b378 WatchSource:0}: Error finding container 1ecd96d51bf49bb7a4e1b000bfdb56a189f71a5d220db560e0eb6219f071b378: Status 404 returned error can't find the container with id 1ecd96d51bf49bb7a4e1b000bfdb56a189f71a5d220db560e0eb6219f071b378
Sep 10 21:12:49.996692 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:49.996654    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc" event={"ID":"257e5f98-8d38-4382-9b1a-6646b033c0a2","Type":"ContainerStarted","Data":"163e456872ef440e87f63d9af22afedb1457deddbd2aafa741acb0a90fbca0b7"}
Sep 10 21:12:49.997225 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:49.996711    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc" event={"ID":"257e5f98-8d38-4382-9b1a-6646b033c0a2","Type":"ContainerStarted","Data":"1ecd96d51bf49bb7a4e1b000bfdb56a189f71a5d220db560e0eb6219f071b378"}
Sep 10 21:12:50.014336 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:50.014275    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc" podStartSLOduration=2.014256695 podStartE2EDuration="2.014256695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:50.01374638 +0000 UTC m=+863.693245230" watchObservedRunningTime="2026-09-10 21:12:50.014256695 +0000 UTC m=+863.693755544"
Sep 10 21:12:51.001822 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:51.001790    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"
Sep 10 21:12:57.336999 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:57.336952    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:12:57.872892 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:57.872848    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:57.873154 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:57.872905    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:57.885499 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:57.885471    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:12:58.052338 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:12:58.052300    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:13:02.010021 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:02.009974    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"
Sep 10 21:13:07.469599 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:07.469561    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:12.835123 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:12.835082    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:13:13.485037 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:13.485001    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:19.584115 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:19.584003    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:19.614510 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:19.614472    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"]
Sep 10 21:13:19.615041 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:19.614971    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc" podUID="257e5f98-8d38-4382-9b1a-6646b033c0a2" containerName="payload-processing" containerID="cri-o://163e456872ef440e87f63d9af22afedb1457deddbd2aafa741acb0a90fbca0b7" gracePeriod=30
Sep 10 21:13:19.684458 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:19.684422    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:19.761656 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:19.761619    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:19.812047 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:19.812010    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:19.899228 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:19.899198    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"
Sep 10 21:13:19.918479 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:19.918441    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:19.953571 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:19.953527    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9nf7\" (UniqueName: \"kubernetes.io/projected/257e5f98-8d38-4382-9b1a-6646b033c0a2-kube-api-access-g9nf7\") pod \"257e5f98-8d38-4382-9b1a-6646b033c0a2\" (UID: \"257e5f98-8d38-4382-9b1a-6646b033c0a2\") "
Sep 10 21:13:19.953778 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:19.953583    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/257e5f98-8d38-4382-9b1a-6646b033c0a2-plugins-config-volume\") pod \"257e5f98-8d38-4382-9b1a-6646b033c0a2\" (UID: \"257e5f98-8d38-4382-9b1a-6646b033c0a2\") "
Sep 10 21:13:19.953972 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:19.953934    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/257e5f98-8d38-4382-9b1a-6646b033c0a2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "257e5f98-8d38-4382-9b1a-6646b033c0a2" (UID: "257e5f98-8d38-4382-9b1a-6646b033c0a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:19.955639 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:19.955614    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/257e5f98-8d38-4382-9b1a-6646b033c0a2-kube-api-access-g9nf7" (OuterVolumeSpecName: "kube-api-access-g9nf7") pod "257e5f98-8d38-4382-9b1a-6646b033c0a2" (UID: "257e5f98-8d38-4382-9b1a-6646b033c0a2"). InnerVolumeSpecName "kube-api-access-g9nf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:13:20.054673 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.054636    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9nf7\" (UniqueName: \"kubernetes.io/projected/257e5f98-8d38-4382-9b1a-6646b033c0a2-kube-api-access-g9nf7\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:20.054673 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.054667    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/257e5f98-8d38-4382-9b1a-6646b033c0a2-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:20.158239 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.158151    2580 generic.go:356] "Generic (PLEG): container finished" podID="257e5f98-8d38-4382-9b1a-6646b033c0a2" containerID="163e456872ef440e87f63d9af22afedb1457deddbd2aafa741acb0a90fbca0b7" exitCode=0
Sep 10 21:13:20.158239 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.158199    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"
Sep 10 21:13:20.158433 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.158241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc" event={"ID":"257e5f98-8d38-4382-9b1a-6646b033c0a2","Type":"ContainerDied","Data":"163e456872ef440e87f63d9af22afedb1457deddbd2aafa741acb0a90fbca0b7"}
Sep 10 21:13:20.158433 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.158301    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc" event={"ID":"257e5f98-8d38-4382-9b1a-6646b033c0a2","Type":"ContainerDied","Data":"1ecd96d51bf49bb7a4e1b000bfdb56a189f71a5d220db560e0eb6219f071b378"}
Sep 10 21:13:20.158433 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.158329    2580 scope.go:117] "RemoveContainer" containerID="163e456872ef440e87f63d9af22afedb1457deddbd2aafa741acb0a90fbca0b7"
Sep 10 21:13:20.169882 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.169858    2580 scope.go:117] "RemoveContainer" containerID="163e456872ef440e87f63d9af22afedb1457deddbd2aafa741acb0a90fbca0b7"
Sep 10 21:13:20.170290 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:13:20.170263    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"163e456872ef440e87f63d9af22afedb1457deddbd2aafa741acb0a90fbca0b7\": container with ID starting with 163e456872ef440e87f63d9af22afedb1457deddbd2aafa741acb0a90fbca0b7 not found: ID does not exist" containerID="163e456872ef440e87f63d9af22afedb1457deddbd2aafa741acb0a90fbca0b7"
Sep 10 21:13:20.170453 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.170310    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"163e456872ef440e87f63d9af22afedb1457deddbd2aafa741acb0a90fbca0b7"} err="failed to get container status \"163e456872ef440e87f63d9af22afedb1457deddbd2aafa741acb0a90fbca0b7\": rpc error: code = NotFound desc = could not find container \"163e456872ef440e87f63d9af22afedb1457deddbd2aafa741acb0a90fbca0b7\": container with ID starting with 163e456872ef440e87f63d9af22afedb1457deddbd2aafa741acb0a90fbca0b7 not found: ID does not exist"
Sep 10 21:13:20.181028 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.180994    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"]
Sep 10 21:13:20.184389 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.184363    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-065d3bbf-5bd8cc96c4-x6btc"]
Sep 10 21:13:20.403089 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.403032    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:20.711889 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.711828    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:20.956332 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.956252    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="257e5f98-8d38-4382-9b1a-6646b033c0a2" path="/var/lib/kubelet/pods/257e5f98-8d38-4382-9b1a-6646b033c0a2/volumes"
Sep 10 21:13:20.959224 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:20.959197    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:22.752222 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:22.752186    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:23.300901 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:23.300857    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:23.401341 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:23.401307    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:24.479753 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:24.479718    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:24.579886 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:24.579846    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:24.739778 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:24.739683    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:24.939899 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:24.939857    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:25.033880 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.033786    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:25.034463 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.034443    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"]
Sep 10 21:13:25.034876 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.034864    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="257e5f98-8d38-4382-9b1a-6646b033c0a2" containerName="payload-processing"
Sep 10 21:13:25.034918 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.034878    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="257e5f98-8d38-4382-9b1a-6646b033c0a2" containerName="payload-processing"
Sep 10 21:13:25.034968 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.034959    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="257e5f98-8d38-4382-9b1a-6646b033c0a2" containerName="payload-processing"
Sep 10 21:13:25.038122 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.038105    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"
Sep 10 21:13:25.040603 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.040582    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-169213ba-dockercfg-8pztg\""
Sep 10 21:13:25.040723 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.040582    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-169213ba\""
Sep 10 21:13:25.047523 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.047495    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"]
Sep 10 21:13:25.075151 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.075116    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"]
Sep 10 21:13:25.078750 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.078732    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"
Sep 10 21:13:25.087428 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.087402    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"]
Sep 10 21:13:25.102478 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.102443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478dfca0-4129-4b5f-a1ae-16833bc36037-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn\" (UID: \"478dfca0-4129-4b5f-a1ae-16833bc36037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"
Sep 10 21:13:25.102678 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.102519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j94c7\" (UniqueName: \"kubernetes.io/projected/478dfca0-4129-4b5f-a1ae-16833bc36037-kube-api-access-j94c7\") pod \"payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn\" (UID: \"478dfca0-4129-4b5f-a1ae-16833bc36037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"
Sep 10 21:13:25.203472 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.203437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478dfca0-4129-4b5f-a1ae-16833bc36037-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn\" (UID: \"478dfca0-4129-4b5f-a1ae-16833bc36037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"
Sep 10 21:13:25.203647 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.203493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c79f6a66-7963-4ef9-9e93-a2d2a7b91785-plugins-config-volume\") pod \"payload-processing-e2e-ait-169213ba-65b6f796df-x8x54\" (UID: \"c79f6a66-7963-4ef9-9e93-a2d2a7b91785\") " pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"
Sep 10 21:13:25.203647 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.203514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j7zl\" (UniqueName: \"kubernetes.io/projected/c79f6a66-7963-4ef9-9e93-a2d2a7b91785-kube-api-access-9j7zl\") pod \"payload-processing-e2e-ait-169213ba-65b6f796df-x8x54\" (UID: \"c79f6a66-7963-4ef9-9e93-a2d2a7b91785\") " pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"
Sep 10 21:13:25.203647 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.203549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j94c7\" (UniqueName: \"kubernetes.io/projected/478dfca0-4129-4b5f-a1ae-16833bc36037-kube-api-access-j94c7\") pod \"payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn\" (UID: \"478dfca0-4129-4b5f-a1ae-16833bc36037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"
Sep 10 21:13:25.204234 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.204213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478dfca0-4129-4b5f-a1ae-16833bc36037-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn\" (UID: \"478dfca0-4129-4b5f-a1ae-16833bc36037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"
Sep 10 21:13:25.211878 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.211835    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j94c7\" (UniqueName: \"kubernetes.io/projected/478dfca0-4129-4b5f-a1ae-16833bc36037-kube-api-access-j94c7\") pod \"payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn\" (UID: \"478dfca0-4129-4b5f-a1ae-16833bc36037\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"
Sep 10 21:13:25.304255 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.304216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c79f6a66-7963-4ef9-9e93-a2d2a7b91785-plugins-config-volume\") pod \"payload-processing-e2e-ait-169213ba-65b6f796df-x8x54\" (UID: \"c79f6a66-7963-4ef9-9e93-a2d2a7b91785\") " pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"
Sep 10 21:13:25.304480 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.304266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j7zl\" (UniqueName: \"kubernetes.io/projected/c79f6a66-7963-4ef9-9e93-a2d2a7b91785-kube-api-access-9j7zl\") pod \"payload-processing-e2e-ait-169213ba-65b6f796df-x8x54\" (UID: \"c79f6a66-7963-4ef9-9e93-a2d2a7b91785\") " pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"
Sep 10 21:13:25.305158 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.305131    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c79f6a66-7963-4ef9-9e93-a2d2a7b91785-plugins-config-volume\") pod \"payload-processing-e2e-ait-169213ba-65b6f796df-x8x54\" (UID: \"c79f6a66-7963-4ef9-9e93-a2d2a7b91785\") " pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"
Sep 10 21:13:25.323148 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.323114    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j7zl\" (UniqueName: \"kubernetes.io/projected/c79f6a66-7963-4ef9-9e93-a2d2a7b91785-kube-api-access-9j7zl\") pod \"payload-processing-e2e-ait-169213ba-65b6f796df-x8x54\" (UID: \"c79f6a66-7963-4ef9-9e93-a2d2a7b91785\") " pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"
Sep 10 21:13:25.349475 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.349440    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"
Sep 10 21:13:25.391076 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.391030    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"
Sep 10 21:13:25.623683 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.623660    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"]
Sep 10 21:13:25.625800 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:13:25.625770    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478dfca0_4129_4b5f_a1ae_16833bc36037.slice/crio-e5d13e211f410b49b6f411fd48b7dea5036391a753c0301125b359560935ece8 WatchSource:0}: Error finding container e5d13e211f410b49b6f411fd48b7dea5036391a753c0301125b359560935ece8: Status 404 returned error can't find the container with id e5d13e211f410b49b6f411fd48b7dea5036391a753c0301125b359560935ece8
Sep 10 21:13:25.679491 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:25.679458    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"]
Sep 10 21:13:25.683372 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:13:25.683336    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc79f6a66_7963_4ef9_9e93_a2d2a7b91785.slice/crio-ee2a1c1c429c5a88bf37d27d6e12f778f0e875d28564cfc05e8a0ca8a6422072 WatchSource:0}: Error finding container ee2a1c1c429c5a88bf37d27d6e12f778f0e875d28564cfc05e8a0ca8a6422072: Status 404 returned error can't find the container with id ee2a1c1c429c5a88bf37d27d6e12f778f0e875d28564cfc05e8a0ca8a6422072
Sep 10 21:13:26.194882 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:26.194843    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54" event={"ID":"c79f6a66-7963-4ef9-9e93-a2d2a7b91785","Type":"ContainerStarted","Data":"adaf4cac948975051ba15176940ee7433a92fb2f47f3e2064466085e85e3bba2"}
Sep 10 21:13:26.195104 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:26.194899    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54" event={"ID":"c79f6a66-7963-4ef9-9e93-a2d2a7b91785","Type":"ContainerStarted","Data":"ee2a1c1c429c5a88bf37d27d6e12f778f0e875d28564cfc05e8a0ca8a6422072"}
Sep 10 21:13:26.195104 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:26.194956    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"
Sep 10 21:13:26.196659 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:26.196634    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn" event={"ID":"478dfca0-4129-4b5f-a1ae-16833bc36037","Type":"ContainerStarted","Data":"926f2318d66afab852752884a508da1cfa7d47d0f83e2cba5a51f3498a47186c"}
Sep 10 21:13:26.196799 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:26.196670    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn" event={"ID":"478dfca0-4129-4b5f-a1ae-16833bc36037","Type":"ContainerStarted","Data":"e5d13e211f410b49b6f411fd48b7dea5036391a753c0301125b359560935ece8"}
Sep 10 21:13:26.196799 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:26.196769    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"
Sep 10 21:13:26.213834 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:26.213773    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54" podStartSLOduration=1.213755255 podStartE2EDuration="1.213755255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:13:26.211492392 +0000 UTC m=+899.890991268" watchObservedRunningTime="2026-09-10 21:13:26.213755255 +0000 UTC m=+899.893254155"
Sep 10 21:13:26.228461 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:26.228373    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn" podStartSLOduration=1.228352743 podStartE2EDuration="1.228352743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:13:26.227478244 +0000 UTC m=+899.906977097" watchObservedRunningTime="2026-09-10 21:13:26.228352743 +0000 UTC m=+899.907851593"
Sep 10 21:13:26.743255 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:26.743185    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"]
Sep 10 21:13:26.778266 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:26.778236    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:32.736277 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:32.736233    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:13:34.447401 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:34.447360    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:34.879870 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:34.879823    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:35.464290 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:35.464254    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:37.204990 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:37.204958    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"
Sep 10 21:13:37.205415 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:37.205075    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"
Sep 10 21:13:41.763842 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:41.763774    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz" podUID="6f234e34-2574-4369-9cd6-e02d3c5da348" containerName="main" containerID="cri-o://d238d7616c8a00b5e7e3eca6a75a31b0879596d6182e9c63e3783cf2a1b8c892" gracePeriod=45
Sep 10 21:13:42.017525 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.017456    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:13:42.071220 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.071181    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-tmp-dir\") pod \"6f234e34-2574-4369-9cd6-e02d3c5da348\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") "
Sep 10 21:13:42.071220 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.071229    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-model-cache\") pod \"6f234e34-2574-4369-9cd6-e02d3c5da348\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") "
Sep 10 21:13:42.071478 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.071248    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-home\") pod \"6f234e34-2574-4369-9cd6-e02d3c5da348\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") "
Sep 10 21:13:42.071478 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.071305    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdfsw\" (UniqueName: \"kubernetes.io/projected/6f234e34-2574-4369-9cd6-e02d3c5da348-kube-api-access-sdfsw\") pod \"6f234e34-2574-4369-9cd6-e02d3c5da348\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") "
Sep 10 21:13:42.071478 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.071338    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-dshm\") pod \"6f234e34-2574-4369-9cd6-e02d3c5da348\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") "
Sep 10 21:13:42.071478 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.071371    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f234e34-2574-4369-9cd6-e02d3c5da348-tls-certs\") pod \"6f234e34-2574-4369-9cd6-e02d3c5da348\" (UID: \"6f234e34-2574-4369-9cd6-e02d3c5da348\") "
Sep 10 21:13:42.071678 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.071482    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f234e34-2574-4369-9cd6-e02d3c5da348" (UID: "6f234e34-2574-4369-9cd6-e02d3c5da348"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:13:42.071678 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.071514    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f234e34-2574-4369-9cd6-e02d3c5da348" (UID: "6f234e34-2574-4369-9cd6-e02d3c5da348"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:13:42.071678 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.071566    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-home" (OuterVolumeSpecName: "home") pod "6f234e34-2574-4369-9cd6-e02d3c5da348" (UID: "6f234e34-2574-4369-9cd6-e02d3c5da348"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:13:42.071787 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.071699    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-home\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:42.071787 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.071716    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-tmp-dir\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:42.071787 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.071728    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-model-cache\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:42.073614 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.073581    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-dshm" (OuterVolumeSpecName: "dshm") pod "6f234e34-2574-4369-9cd6-e02d3c5da348" (UID: "6f234e34-2574-4369-9cd6-e02d3c5da348"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:13:42.073726 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.073635    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f234e34-2574-4369-9cd6-e02d3c5da348-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f234e34-2574-4369-9cd6-e02d3c5da348" (UID: "6f234e34-2574-4369-9cd6-e02d3c5da348"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:42.073726 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.073643    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f234e34-2574-4369-9cd6-e02d3c5da348-kube-api-access-sdfsw" (OuterVolumeSpecName: "kube-api-access-sdfsw") pod "6f234e34-2574-4369-9cd6-e02d3c5da348" (UID: "6f234e34-2574-4369-9cd6-e02d3c5da348"). InnerVolumeSpecName "kube-api-access-sdfsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:13:42.172421 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.172381    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdfsw\" (UniqueName: \"kubernetes.io/projected/6f234e34-2574-4369-9cd6-e02d3c5da348-kube-api-access-sdfsw\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:42.172421 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.172413    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f234e34-2574-4369-9cd6-e02d3c5da348-dshm\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:42.172421 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.172424    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f234e34-2574-4369-9cd6-e02d3c5da348-tls-certs\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:42.285047 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.284953    2580 generic.go:356] "Generic (PLEG): container finished" podID="6f234e34-2574-4369-9cd6-e02d3c5da348" containerID="d238d7616c8a00b5e7e3eca6a75a31b0879596d6182e9c63e3783cf2a1b8c892" exitCode=0
Sep 10 21:13:42.285047 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.284994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz" event={"ID":"6f234e34-2574-4369-9cd6-e02d3c5da348","Type":"ContainerDied","Data":"d238d7616c8a00b5e7e3eca6a75a31b0879596d6182e9c63e3783cf2a1b8c892"}
Sep 10 21:13:42.285047 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.285000    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"
Sep 10 21:13:42.285047 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.285024    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz" event={"ID":"6f234e34-2574-4369-9cd6-e02d3c5da348","Type":"ContainerDied","Data":"4a135deb64f207942630b21233994e4fb127e864858cf43c28cbd42eefb5644a"}
Sep 10 21:13:42.285047 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.285041    2580 scope.go:117] "RemoveContainer" containerID="d238d7616c8a00b5e7e3eca6a75a31b0879596d6182e9c63e3783cf2a1b8c892"
Sep 10 21:13:42.297491 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.297467    2580 scope.go:117] "RemoveContainer" containerID="d238d7616c8a00b5e7e3eca6a75a31b0879596d6182e9c63e3783cf2a1b8c892"
Sep 10 21:13:42.297803 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:13:42.297771    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d238d7616c8a00b5e7e3eca6a75a31b0879596d6182e9c63e3783cf2a1b8c892\": container with ID starting with d238d7616c8a00b5e7e3eca6a75a31b0879596d6182e9c63e3783cf2a1b8c892 not found: ID does not exist" containerID="d238d7616c8a00b5e7e3eca6a75a31b0879596d6182e9c63e3783cf2a1b8c892"
Sep 10 21:13:42.297860 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.297800    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d238d7616c8a00b5e7e3eca6a75a31b0879596d6182e9c63e3783cf2a1b8c892"} err="failed to get container status \"d238d7616c8a00b5e7e3eca6a75a31b0879596d6182e9c63e3783cf2a1b8c892\": rpc error: code = NotFound desc = could not find container \"d238d7616c8a00b5e7e3eca6a75a31b0879596d6182e9c63e3783cf2a1b8c892\": container with ID starting with d238d7616c8a00b5e7e3eca6a75a31b0879596d6182e9c63e3783cf2a1b8c892 not found: ID does not exist"
Sep 10 21:13:42.308450 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.308413    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"]
Sep 10 21:13:42.312463 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.312437    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-1c65017b-kserve-ddfbcf9cd-d58fz"]
Sep 10 21:13:42.957522 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:42.957476    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f234e34-2574-4369-9cd6-e02d3c5da348" path="/var/lib/kubelet/pods/6f234e34-2574-4369-9cd6-e02d3c5da348/volumes"
Sep 10 21:13:45.049446 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.049417    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:45.078633 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.078598    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"]
Sep 10 21:13:45.078866 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.078838    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz" podUID="b41f6c23-c31f-4cd7-8cd0-608c61ecbd60" containerName="payload-processing" containerID="cri-o://a77a8b02cd4ac804b1685885d0f46060ddf34fe7ee1d84df83786d779bb3b189" gracePeriod=30
Sep 10 21:13:45.130161 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.130126    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:45.230960 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.230927    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:45.316163 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.316051    2580 generic.go:356] "Generic (PLEG): container finished" podID="b41f6c23-c31f-4cd7-8cd0-608c61ecbd60" containerID="a77a8b02cd4ac804b1685885d0f46060ddf34fe7ee1d84df83786d779bb3b189" exitCode=0
Sep 10 21:13:45.316163 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.316100    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz" event={"ID":"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60","Type":"ContainerDied","Data":"a77a8b02cd4ac804b1685885d0f46060ddf34fe7ee1d84df83786d779bb3b189"}
Sep 10 21:13:45.329756 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.329723    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:45.352825 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.352793    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"
Sep 10 21:13:45.411302 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.411263    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b41f6c23-c31f-4cd7-8cd0-608c61ecbd60-plugins-config-volume\") pod \"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60\" (UID: \"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60\") "
Sep 10 21:13:45.411499 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.411362    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9zgw\" (UniqueName: \"kubernetes.io/projected/b41f6c23-c31f-4cd7-8cd0-608c61ecbd60-kube-api-access-v9zgw\") pod \"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60\" (UID: \"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60\") "
Sep 10 21:13:45.411661 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.411639    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b41f6c23-c31f-4cd7-8cd0-608c61ecbd60-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b41f6c23-c31f-4cd7-8cd0-608c61ecbd60" (UID: "b41f6c23-c31f-4cd7-8cd0-608c61ecbd60"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:45.413550 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.413528    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b41f6c23-c31f-4cd7-8cd0-608c61ecbd60-kube-api-access-v9zgw" (OuterVolumeSpecName: "kube-api-access-v9zgw") pod "b41f6c23-c31f-4cd7-8cd0-608c61ecbd60" (UID: "b41f6c23-c31f-4cd7-8cd0-608c61ecbd60"). InnerVolumeSpecName "kube-api-access-v9zgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:13:45.429757 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.429725    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:45.512639 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.512598    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9zgw\" (UniqueName: \"kubernetes.io/projected/b41f6c23-c31f-4cd7-8cd0-608c61ecbd60-kube-api-access-v9zgw\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:45.512639 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.512634    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b41f6c23-c31f-4cd7-8cd0-608c61ecbd60-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:45.530431 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.530396    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:45.714833 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:45.714752    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:46.322775 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:46.322746    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"
Sep 10 21:13:46.323283 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:46.322741    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz" event={"ID":"b41f6c23-c31f-4cd7-8cd0-608c61ecbd60","Type":"ContainerDied","Data":"965d0f0912db236267c24fac2695ae21f37fa4287e2e3d04c10666bd1097fefc"}
Sep 10 21:13:46.323283 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:46.322895    2580 scope.go:117] "RemoveContainer" containerID="a77a8b02cd4ac804b1685885d0f46060ddf34fe7ee1d84df83786d779bb3b189"
Sep 10 21:13:46.344979 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:46.344941    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"]
Sep 10 21:13:46.348209 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:46.348178    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-5e2fa5-7d6bc885f7-8m5hz"]
Sep 10 21:13:46.956551 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:46.956517    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b41f6c23-c31f-4cd7-8cd0-608c61ecbd60" path="/var/lib/kubelet/pods/b41f6c23-c31f-4cd7-8cd0-608c61ecbd60/volumes"
Sep 10 21:13:52.261742 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:52.261700    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:52.361900 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:52.361868    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:52.539394 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:52.539363    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:53.025516 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:53.025423    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:53.333569 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:53.333529    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:53.434497 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:53.434462    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:54.331220 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:54.331183    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:13:55.075939 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.075899    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:55.110735 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.110699    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"]
Sep 10 21:13:55.111272 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.111212    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54" podUID="c79f6a66-7963-4ef9-9e93-a2d2a7b91785" containerName="payload-processing" containerID="cri-o://adaf4cac948975051ba15176940ee7433a92fb2f47f3e2064466085e85e3bba2" gracePeriod=30
Sep 10 21:13:55.119526 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.119495    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"]
Sep 10 21:13:55.119828 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.119778    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn" podUID="478dfca0-4129-4b5f-a1ae-16833bc36037" containerName="payload-pre-processing" containerID="cri-o://926f2318d66afab852752884a508da1cfa7d47d0f83e2cba5a51f3498a47186c" gracePeriod=30
Sep 10 21:13:55.153948 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.153909    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:55.222833 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.222794    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:55.285678 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.285645    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:55.344085 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.344004    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:55.375942 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.375911    2580 generic.go:356] "Generic (PLEG): container finished" podID="c79f6a66-7963-4ef9-9e93-a2d2a7b91785" containerID="adaf4cac948975051ba15176940ee7433a92fb2f47f3e2064466085e85e3bba2" exitCode=0
Sep 10 21:13:55.376171 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.375992    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54" event={"ID":"c79f6a66-7963-4ef9-9e93-a2d2a7b91785","Type":"ContainerDied","Data":"adaf4cac948975051ba15176940ee7433a92fb2f47f3e2064466085e85e3bba2"}
Sep 10 21:13:55.377850 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.377825    2580 generic.go:356] "Generic (PLEG): container finished" podID="478dfca0-4129-4b5f-a1ae-16833bc36037" containerID="926f2318d66afab852752884a508da1cfa7d47d0f83e2cba5a51f3498a47186c" exitCode=0
Sep 10 21:13:55.377979 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.377894    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn" event={"ID":"478dfca0-4129-4b5f-a1ae-16833bc36037","Type":"ContainerDied","Data":"926f2318d66afab852752884a508da1cfa7d47d0f83e2cba5a51f3498a47186c"}
Sep 10 21:13:55.408917 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.408891    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"
Sep 10 21:13:55.412095 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.412072    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"
Sep 10 21:13:55.509592 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.509545    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j94c7\" (UniqueName: \"kubernetes.io/projected/478dfca0-4129-4b5f-a1ae-16833bc36037-kube-api-access-j94c7\") pod \"478dfca0-4129-4b5f-a1ae-16833bc36037\" (UID: \"478dfca0-4129-4b5f-a1ae-16833bc36037\") "
Sep 10 21:13:55.509799 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.509646    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478dfca0-4129-4b5f-a1ae-16833bc36037-plugins-config-volume\") pod \"478dfca0-4129-4b5f-a1ae-16833bc36037\" (UID: \"478dfca0-4129-4b5f-a1ae-16833bc36037\") "
Sep 10 21:13:55.509799 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.509688    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9j7zl\" (UniqueName: \"kubernetes.io/projected/c79f6a66-7963-4ef9-9e93-a2d2a7b91785-kube-api-access-9j7zl\") pod \"c79f6a66-7963-4ef9-9e93-a2d2a7b91785\" (UID: \"c79f6a66-7963-4ef9-9e93-a2d2a7b91785\") "
Sep 10 21:13:55.509799 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.509755    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c79f6a66-7963-4ef9-9e93-a2d2a7b91785-plugins-config-volume\") pod \"c79f6a66-7963-4ef9-9e93-a2d2a7b91785\" (UID: \"c79f6a66-7963-4ef9-9e93-a2d2a7b91785\") "
Sep 10 21:13:55.510145 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.510117    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/478dfca0-4129-4b5f-a1ae-16833bc36037-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "478dfca0-4129-4b5f-a1ae-16833bc36037" (UID: "478dfca0-4129-4b5f-a1ae-16833bc36037"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:55.510286 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.510205    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c79f6a66-7963-4ef9-9e93-a2d2a7b91785-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c79f6a66-7963-4ef9-9e93-a2d2a7b91785" (UID: "c79f6a66-7963-4ef9-9e93-a2d2a7b91785"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:55.511818 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.511780    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/478dfca0-4129-4b5f-a1ae-16833bc36037-kube-api-access-j94c7" (OuterVolumeSpecName: "kube-api-access-j94c7") pod "478dfca0-4129-4b5f-a1ae-16833bc36037" (UID: "478dfca0-4129-4b5f-a1ae-16833bc36037"). InnerVolumeSpecName "kube-api-access-j94c7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:13:55.511961 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.511933    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c79f6a66-7963-4ef9-9e93-a2d2a7b91785-kube-api-access-9j7zl" (OuterVolumeSpecName: "kube-api-access-9j7zl") pod "c79f6a66-7963-4ef9-9e93-a2d2a7b91785" (UID: "c79f6a66-7963-4ef9-9e93-a2d2a7b91785"). InnerVolumeSpecName "kube-api-access-9j7zl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:13:55.611092 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.610984    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c79f6a66-7963-4ef9-9e93-a2d2a7b91785-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:55.611092 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.611015    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j94c7\" (UniqueName: \"kubernetes.io/projected/478dfca0-4129-4b5f-a1ae-16833bc36037-kube-api-access-j94c7\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:55.611092 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.611028    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478dfca0-4129-4b5f-a1ae-16833bc36037-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:55.611092 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:55.611039    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j7zl\" (UniqueName: \"kubernetes.io/projected/c79f6a66-7963-4ef9-9e93-a2d2a7b91785-kube-api-access-9j7zl\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:13:56.384756 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.384722    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"
Sep 10 21:13:56.385257 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.384721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54" event={"ID":"c79f6a66-7963-4ef9-9e93-a2d2a7b91785","Type":"ContainerDied","Data":"ee2a1c1c429c5a88bf37d27d6e12f778f0e875d28564cfc05e8a0ca8a6422072"}
Sep 10 21:13:56.385257 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.384895    2580 scope.go:117] "RemoveContainer" containerID="adaf4cac948975051ba15176940ee7433a92fb2f47f3e2064466085e85e3bba2"
Sep 10 21:13:56.386373 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.386208    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"
Sep 10 21:13:56.386373 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.386257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn" event={"ID":"478dfca0-4129-4b5f-a1ae-16833bc36037","Type":"ContainerDied","Data":"e5d13e211f410b49b6f411fd48b7dea5036391a753c0301125b359560935ece8"}
Sep 10 21:13:56.395490 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.395466    2580 scope.go:117] "RemoveContainer" containerID="926f2318d66afab852752884a508da1cfa7d47d0f83e2cba5a51f3498a47186c"
Sep 10 21:13:56.411787 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.411757    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"]
Sep 10 21:13:56.417748 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.417718    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-169213ba-75f86d89d6-zxksn"]
Sep 10 21:13:56.428518 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.428482    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"]
Sep 10 21:13:56.432713 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.432683    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-169213ba-65b6f796df-x8x54"]
Sep 10 21:13:56.476595 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.476560    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:56.934087 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.934025    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:56.957045 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.957012    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="478dfca0-4129-4b5f-a1ae-16833bc36037" path="/var/lib/kubelet/pods/478dfca0-4129-4b5f-a1ae-16833bc36037/volumes"
Sep 10 21:13:56.957469 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:56.957452    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c79f6a66-7963-4ef9-9e93-a2d2a7b91785" path="/var/lib/kubelet/pods/c79f6a66-7963-4ef9-9e93-a2d2a7b91785/volumes"
Sep 10 21:13:57.251391 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:57.251295    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:57.315926 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:57.315892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:57.415387 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:57.415354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:58.625053 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:58.625014    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:58.722291 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:58.722252    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:58.910478 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:58.910394    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:59.101370 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.101333    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:59.201950 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.201864    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:59.238323 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.238287    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"]
Sep 10 21:13:59.238723 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.238710    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c79f6a66-7963-4ef9-9e93-a2d2a7b91785" containerName="payload-processing"
Sep 10 21:13:59.238775 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.238725    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c79f6a66-7963-4ef9-9e93-a2d2a7b91785" containerName="payload-processing"
Sep 10 21:13:59.238775 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.238739    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="478dfca0-4129-4b5f-a1ae-16833bc36037" containerName="payload-pre-processing"
Sep 10 21:13:59.238775 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.238745    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="478dfca0-4129-4b5f-a1ae-16833bc36037" containerName="payload-pre-processing"
Sep 10 21:13:59.238775 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.238756    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f234e34-2574-4369-9cd6-e02d3c5da348" containerName="main"
Sep 10 21:13:59.238775 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.238762    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f234e34-2574-4369-9cd6-e02d3c5da348" containerName="main"
Sep 10 21:13:59.238775 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.238773    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b41f6c23-c31f-4cd7-8cd0-608c61ecbd60" containerName="payload-processing"
Sep 10 21:13:59.238992 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.238778    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b41f6c23-c31f-4cd7-8cd0-608c61ecbd60" containerName="payload-processing"
Sep 10 21:13:59.238992 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.238840    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b41f6c23-c31f-4cd7-8cd0-608c61ecbd60" containerName="payload-processing"
Sep 10 21:13:59.238992 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.238851    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f234e34-2574-4369-9cd6-e02d3c5da348" containerName="main"
Sep 10 21:13:59.238992 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.238859    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="478dfca0-4129-4b5f-a1ae-16833bc36037" containerName="payload-pre-processing"
Sep 10 21:13:59.238992 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.238865    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c79f6a66-7963-4ef9-9e93-a2d2a7b91785" containerName="payload-processing"
Sep 10 21:13:59.243446 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.243420    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"
Sep 10 21:13:59.246136 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.246111    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-2bbc78\""
Sep 10 21:13:59.246584 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.246562    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-2bbc78-dockercfg-4zd7b\""
Sep 10 21:13:59.252230 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.252208    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"]
Sep 10 21:13:59.345074 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.345026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/832cd765-2f27-4a69-a9a2-18581ae6a084-plugins-config-volume\") pod \"payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd\" (UID: \"832cd765-2f27-4a69-a9a2-18581ae6a084\") " pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"
Sep 10 21:13:59.345252 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.345153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpw96\" (UniqueName: \"kubernetes.io/projected/832cd765-2f27-4a69-a9a2-18581ae6a084-kube-api-access-kpw96\") pod \"payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd\" (UID: \"832cd765-2f27-4a69-a9a2-18581ae6a084\") " pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"
Sep 10 21:13:59.445947 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.445905    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/832cd765-2f27-4a69-a9a2-18581ae6a084-plugins-config-volume\") pod \"payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd\" (UID: \"832cd765-2f27-4a69-a9a2-18581ae6a084\") " pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"
Sep 10 21:13:59.446184 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.445984    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpw96\" (UniqueName: \"kubernetes.io/projected/832cd765-2f27-4a69-a9a2-18581ae6a084-kube-api-access-kpw96\") pod \"payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd\" (UID: \"832cd765-2f27-4a69-a9a2-18581ae6a084\") " pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"
Sep 10 21:13:59.446714 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.446690    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/832cd765-2f27-4a69-a9a2-18581ae6a084-plugins-config-volume\") pod \"payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd\" (UID: \"832cd765-2f27-4a69-a9a2-18581ae6a084\") " pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"
Sep 10 21:13:59.454449 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.454374    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpw96\" (UniqueName: \"kubernetes.io/projected/832cd765-2f27-4a69-a9a2-18581ae6a084-kube-api-access-kpw96\") pod \"payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd\" (UID: \"832cd765-2f27-4a69-a9a2-18581ae6a084\") " pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"
Sep 10 21:13:59.555317 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.555274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"
Sep 10 21:13:59.689010 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.688969    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:59.784672 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.784557    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:13:59.837637 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:13:59.837611    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"]
Sep 10 21:13:59.840412 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:13:59.840380    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod832cd765_2f27_4a69_a9a2_18581ae6a084.slice/crio-5f64ff99d0325574bd4c11436b34590bba33b74a7cab35fa20c0a9c0d35fd528 WatchSource:0}: Error finding container 5f64ff99d0325574bd4c11436b34590bba33b74a7cab35fa20c0a9c0d35fd528: Status 404 returned error can't find the container with id 5f64ff99d0325574bd4c11436b34590bba33b74a7cab35fa20c0a9c0d35fd528
Sep 10 21:14:00.411581 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:00.411540    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd" event={"ID":"832cd765-2f27-4a69-a9a2-18581ae6a084","Type":"ContainerStarted","Data":"70057923e17c514a1083ed1251771e6963f70013b77347712a050861d6f9a5a6"}
Sep 10 21:14:00.411782 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:00.411657    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"
Sep 10 21:14:00.411782 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:00.411683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd" event={"ID":"832cd765-2f27-4a69-a9a2-18581ae6a084","Type":"ContainerStarted","Data":"5f64ff99d0325574bd4c11436b34590bba33b74a7cab35fa20c0a9c0d35fd528"}
Sep 10 21:14:00.429498 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:00.429438    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd" podStartSLOduration=1.429418699 podStartE2EDuration="1.429418699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:14:00.426827114 +0000 UTC m=+934.106325970" watchObservedRunningTime="2026-09-10 21:14:00.429418699 +0000 UTC m=+934.108917549"
Sep 10 21:14:00.868717 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:00.868654    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:00.931825 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:00.931793    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:01.062331 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.062293    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:01.350587 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.350556    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:01.419792 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.419751    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:01.420991 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.420971    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"]
Sep 10 21:14:01.424833 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.424811    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"
Sep 10 21:14:01.427262 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.427240    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-3002a381\""
Sep 10 21:14:01.433536 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.433496    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"]
Sep 10 21:14:01.461901 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.461859    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"]
Sep 10 21:14:01.466402 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.466373    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"
Sep 10 21:14:01.468920 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.468891    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-3002a381-dockercfg-l7mrn\""
Sep 10 21:14:01.476012 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.475980    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"]
Sep 10 21:14:01.566794 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.566756    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x7rg\" (UniqueName: \"kubernetes.io/projected/360ae7ff-4219-4a46-af42-c24c9a2f961e-kube-api-access-9x7rg\") pod \"payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn\" (UID: \"360ae7ff-4219-4a46-af42-c24c9a2f961e\") " pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"
Sep 10 21:14:01.566961 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.566872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/360ae7ff-4219-4a46-af42-c24c9a2f961e-plugins-config-volume\") pod \"payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn\" (UID: \"360ae7ff-4219-4a46-af42-c24c9a2f961e\") " pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"
Sep 10 21:14:01.566961 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.566949    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5571f5ec-1c18-4ae6-8149-47407e0b8df1-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc\" (UID: \"5571f5ec-1c18-4ae6-8149-47407e0b8df1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"
Sep 10 21:14:01.567099 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.567039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktc5b\" (UniqueName: \"kubernetes.io/projected/5571f5ec-1c18-4ae6-8149-47407e0b8df1-kube-api-access-ktc5b\") pod \"payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc\" (UID: \"5571f5ec-1c18-4ae6-8149-47407e0b8df1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"
Sep 10 21:14:01.667810 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.667704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktc5b\" (UniqueName: \"kubernetes.io/projected/5571f5ec-1c18-4ae6-8149-47407e0b8df1-kube-api-access-ktc5b\") pod \"payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc\" (UID: \"5571f5ec-1c18-4ae6-8149-47407e0b8df1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"
Sep 10 21:14:01.667810 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.667773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x7rg\" (UniqueName: \"kubernetes.io/projected/360ae7ff-4219-4a46-af42-c24c9a2f961e-kube-api-access-9x7rg\") pod \"payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn\" (UID: \"360ae7ff-4219-4a46-af42-c24c9a2f961e\") " pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"
Sep 10 21:14:01.668028 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.667829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/360ae7ff-4219-4a46-af42-c24c9a2f961e-plugins-config-volume\") pod \"payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn\" (UID: \"360ae7ff-4219-4a46-af42-c24c9a2f961e\") " pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"
Sep 10 21:14:01.668028 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.667964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5571f5ec-1c18-4ae6-8149-47407e0b8df1-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc\" (UID: \"5571f5ec-1c18-4ae6-8149-47407e0b8df1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"
Sep 10 21:14:01.668601 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.668581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/360ae7ff-4219-4a46-af42-c24c9a2f961e-plugins-config-volume\") pod \"payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn\" (UID: \"360ae7ff-4219-4a46-af42-c24c9a2f961e\") " pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"
Sep 10 21:14:01.668640 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.668611    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5571f5ec-1c18-4ae6-8149-47407e0b8df1-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc\" (UID: \"5571f5ec-1c18-4ae6-8149-47407e0b8df1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"
Sep 10 21:14:01.676205 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.676181    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x7rg\" (UniqueName: \"kubernetes.io/projected/360ae7ff-4219-4a46-af42-c24c9a2f961e-kube-api-access-9x7rg\") pod \"payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn\" (UID: \"360ae7ff-4219-4a46-af42-c24c9a2f961e\") " pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"
Sep 10 21:14:01.676323 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.676207    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktc5b\" (UniqueName: \"kubernetes.io/projected/5571f5ec-1c18-4ae6-8149-47407e0b8df1-kube-api-access-ktc5b\") pod \"payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc\" (UID: \"5571f5ec-1c18-4ae6-8149-47407e0b8df1\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"
Sep 10 21:14:01.737148 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.737108    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"
Sep 10 21:14:01.779149 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:01.779107    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"
Sep 10 21:14:02.011723 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:02.011694    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"]
Sep 10 21:14:02.013524 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:14:02.013492    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5571f5ec_1c18_4ae6_8149_47407e0b8df1.slice/crio-b4e2336dc13968737328f671e6b0b5695d428b96cd548cf348008352986b6ed2 WatchSource:0}: Error finding container b4e2336dc13968737328f671e6b0b5695d428b96cd548cf348008352986b6ed2: Status 404 returned error can't find the container with id b4e2336dc13968737328f671e6b0b5695d428b96cd548cf348008352986b6ed2
Sep 10 21:14:02.064891 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:02.064861    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"]
Sep 10 21:14:02.066882 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:14:02.066854    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod360ae7ff_4219_4a46_af42_c24c9a2f961e.slice/crio-84d489e2322e3f811396b847eca8dc239b06e2d403149ba5d0508a7752dfc377 WatchSource:0}: Error finding container 84d489e2322e3f811396b847eca8dc239b06e2d403149ba5d0508a7752dfc377: Status 404 returned error can't find the container with id 84d489e2322e3f811396b847eca8dc239b06e2d403149ba5d0508a7752dfc377
Sep 10 21:14:02.425577 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:02.425527    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn" event={"ID":"360ae7ff-4219-4a46-af42-c24c9a2f961e","Type":"ContainerStarted","Data":"4ef2eddf667d0ac902cc8998be1feaf503872d6c5f0360ce385c4a245632646a"}
Sep 10 21:14:02.425793 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:02.425598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn" event={"ID":"360ae7ff-4219-4a46-af42-c24c9a2f961e","Type":"ContainerStarted","Data":"84d489e2322e3f811396b847eca8dc239b06e2d403149ba5d0508a7752dfc377"}
Sep 10 21:14:02.425793 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:02.425627    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"
Sep 10 21:14:02.427274 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:02.427245    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc" event={"ID":"5571f5ec-1c18-4ae6-8149-47407e0b8df1","Type":"ContainerStarted","Data":"b5ce011b89e76d6d2e0fff023a14b1464b935c88690a9455b0565904b49297a7"}
Sep 10 21:14:02.427395 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:02.427295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc" event={"ID":"5571f5ec-1c18-4ae6-8149-47407e0b8df1","Type":"ContainerStarted","Data":"b4e2336dc13968737328f671e6b0b5695d428b96cd548cf348008352986b6ed2"}
Sep 10 21:14:02.427395 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:02.427326    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"
Sep 10 21:14:02.443166 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:02.443096    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn" podStartSLOduration=1.4430458879999999 podStartE2EDuration="1.443045888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:14:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:14:02.442293296 +0000 UTC m=+936.121792147" watchObservedRunningTime="2026-09-10 21:14:02.443045888 +0000 UTC m=+936.122544740"
Sep 10 21:14:02.458768 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:02.458704    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc" podStartSLOduration=1.458684697 podStartE2EDuration="1.458684697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:14:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:14:02.457251901 +0000 UTC m=+936.136750743" watchObservedRunningTime="2026-09-10 21:14:02.458684697 +0000 UTC m=+936.138183550"
Sep 10 21:14:04.574871 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:04.574823    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:11.419870 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:11.419833    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"
Sep 10 21:14:13.436497 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:13.436468    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"
Sep 10 21:14:13.436856 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:13.436534    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"
Sep 10 21:14:14.536305 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:14.536274    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:14:19.556487 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:19.556455    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:21.699735 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:21.699704    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:23.112518 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:23.112471    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:29.246041 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:29.246008    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:14:31.406111 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.406050    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:31.435752 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.435719    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"]
Sep 10 21:14:31.436035 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.435983    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn" podUID="360ae7ff-4219-4a46-af42-c24c9a2f961e" containerName="payload-processing" containerID="cri-o://4ef2eddf667d0ac902cc8998be1feaf503872d6c5f0360ce385c4a245632646a" gracePeriod=30
Sep 10 21:14:31.443733 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.443707    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"]
Sep 10 21:14:31.443958 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.443914    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc" podUID="5571f5ec-1c18-4ae6-8149-47407e0b8df1" containerName="payload-pre-processing" containerID="cri-o://b5ce011b89e76d6d2e0fff023a14b1464b935c88690a9455b0565904b49297a7" gracePeriod=30
Sep 10 21:14:31.468660 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.468632    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:31.572101 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.572020    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:31.588136 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.588106    2580 generic.go:356] "Generic (PLEG): container finished" podID="5571f5ec-1c18-4ae6-8149-47407e0b8df1" containerID="b5ce011b89e76d6d2e0fff023a14b1464b935c88690a9455b0565904b49297a7" exitCode=0
Sep 10 21:14:31.588280 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.588251    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc" event={"ID":"5571f5ec-1c18-4ae6-8149-47407e0b8df1","Type":"ContainerDied","Data":"b5ce011b89e76d6d2e0fff023a14b1464b935c88690a9455b0565904b49297a7"}
Sep 10 21:14:31.590509 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.590480    2580 generic.go:356] "Generic (PLEG): container finished" podID="360ae7ff-4219-4a46-af42-c24c9a2f961e" containerID="4ef2eddf667d0ac902cc8998be1feaf503872d6c5f0360ce385c4a245632646a" exitCode=0
Sep 10 21:14:31.590652 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.590551    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn" event={"ID":"360ae7ff-4219-4a46-af42-c24c9a2f961e","Type":"ContainerDied","Data":"4ef2eddf667d0ac902cc8998be1feaf503872d6c5f0360ce385c4a245632646a"}
Sep 10 21:14:31.626447 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.626397    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:31.724249 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.724228    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"
Sep 10 21:14:31.727156 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.727128    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"
Sep 10 21:14:31.771970 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.771939    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:31.819183 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.819154    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktc5b\" (UniqueName: \"kubernetes.io/projected/5571f5ec-1c18-4ae6-8149-47407e0b8df1-kube-api-access-ktc5b\") pod \"5571f5ec-1c18-4ae6-8149-47407e0b8df1\" (UID: \"5571f5ec-1c18-4ae6-8149-47407e0b8df1\") "
Sep 10 21:14:31.819295 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.819236    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9x7rg\" (UniqueName: \"kubernetes.io/projected/360ae7ff-4219-4a46-af42-c24c9a2f961e-kube-api-access-9x7rg\") pod \"360ae7ff-4219-4a46-af42-c24c9a2f961e\" (UID: \"360ae7ff-4219-4a46-af42-c24c9a2f961e\") "
Sep 10 21:14:31.819418 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.819398    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5571f5ec-1c18-4ae6-8149-47407e0b8df1-plugins-config-volume\") pod \"5571f5ec-1c18-4ae6-8149-47407e0b8df1\" (UID: \"5571f5ec-1c18-4ae6-8149-47407e0b8df1\") "
Sep 10 21:14:31.819522 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.819505    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/360ae7ff-4219-4a46-af42-c24c9a2f961e-plugins-config-volume\") pod \"360ae7ff-4219-4a46-af42-c24c9a2f961e\" (UID: \"360ae7ff-4219-4a46-af42-c24c9a2f961e\") "
Sep 10 21:14:31.819716 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.819693    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5571f5ec-1c18-4ae6-8149-47407e0b8df1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5571f5ec-1c18-4ae6-8149-47407e0b8df1" (UID: "5571f5ec-1c18-4ae6-8149-47407e0b8df1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:14:31.819840 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.819808    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5571f5ec-1c18-4ae6-8149-47407e0b8df1-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:14:31.819916 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.819878    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/360ae7ff-4219-4a46-af42-c24c9a2f961e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "360ae7ff-4219-4a46-af42-c24c9a2f961e" (UID: "360ae7ff-4219-4a46-af42-c24c9a2f961e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:14:31.821166 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.821147    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/360ae7ff-4219-4a46-af42-c24c9a2f961e-kube-api-access-9x7rg" (OuterVolumeSpecName: "kube-api-access-9x7rg") pod "360ae7ff-4219-4a46-af42-c24c9a2f961e" (UID: "360ae7ff-4219-4a46-af42-c24c9a2f961e"). InnerVolumeSpecName "kube-api-access-9x7rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:14:31.821239 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.821221    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5571f5ec-1c18-4ae6-8149-47407e0b8df1-kube-api-access-ktc5b" (OuterVolumeSpecName: "kube-api-access-ktc5b") pod "5571f5ec-1c18-4ae6-8149-47407e0b8df1" (UID: "5571f5ec-1c18-4ae6-8149-47407e0b8df1"). InnerVolumeSpecName "kube-api-access-ktc5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:14:31.920726 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.920696    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/360ae7ff-4219-4a46-af42-c24c9a2f961e-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:14:31.920726 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.920724    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktc5b\" (UniqueName: \"kubernetes.io/projected/5571f5ec-1c18-4ae6-8149-47407e0b8df1-kube-api-access-ktc5b\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:14:31.920909 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:31.920739    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9x7rg\" (UniqueName: \"kubernetes.io/projected/360ae7ff-4219-4a46-af42-c24c9a2f961e-kube-api-access-9x7rg\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:14:32.474527 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.474487    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:32.534801 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.534768    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:32.595326 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.595298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:32.597426 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.597404    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"
Sep 10 21:14:32.597540 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.597434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc" event={"ID":"5571f5ec-1c18-4ae6-8149-47407e0b8df1","Type":"ContainerDied","Data":"b4e2336dc13968737328f671e6b0b5695d428b96cd548cf348008352986b6ed2"}
Sep 10 21:14:32.597540 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.597494    2580 scope.go:117] "RemoveContainer" containerID="b5ce011b89e76d6d2e0fff023a14b1464b935c88690a9455b0565904b49297a7"
Sep 10 21:14:32.598939 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.598921    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"
Sep 10 21:14:32.599025 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.598919    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn" event={"ID":"360ae7ff-4219-4a46-af42-c24c9a2f961e","Type":"ContainerDied","Data":"84d489e2322e3f811396b847eca8dc239b06e2d403149ba5d0508a7752dfc377"}
Sep 10 21:14:32.608152 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.608135    2580 scope.go:117] "RemoveContainer" containerID="4ef2eddf667d0ac902cc8998be1feaf503872d6c5f0360ce385c4a245632646a"
Sep 10 21:14:32.623371 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.623349    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"]
Sep 10 21:14:32.629099 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.629073    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3002a381-64d48b9fdb-t6zgc"]
Sep 10 21:14:32.644458 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.644438    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"]
Sep 10 21:14:32.649608 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.649586    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3002a381-6644bc9dc7-6b2pn"]
Sep 10 21:14:32.956155 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.956124    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="360ae7ff-4219-4a46-af42-c24c9a2f961e" path="/var/lib/kubelet/pods/360ae7ff-4219-4a46-af42-c24c9a2f961e/volumes"
Sep 10 21:14:32.956480 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:32.956467    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5571f5ec-1c18-4ae6-8149-47407e0b8df1" path="/var/lib/kubelet/pods/5571f5ec-1c18-4ae6-8149-47407e0b8df1/volumes"
Sep 10 21:14:34.098535 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:34.098501    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:40.862324 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:40.862287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:40.960488 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:40.960453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:41.058456 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:41.058424    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:41.157245 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:41.157147    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:41.559909 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:41.559866    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:41.900625 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:41.900535    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:14:43.149202 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:43.149164    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:14:56.045675 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:14:56.045590    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:15:00.146355 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.146322    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817915-thzld"]
Sep 10 21:15:00.146811 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.146766    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="360ae7ff-4219-4a46-af42-c24c9a2f961e" containerName="payload-processing"
Sep 10 21:15:00.146811 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.146780    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="360ae7ff-4219-4a46-af42-c24c9a2f961e" containerName="payload-processing"
Sep 10 21:15:00.146811 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.146795    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5571f5ec-1c18-4ae6-8149-47407e0b8df1" containerName="payload-pre-processing"
Sep 10 21:15:00.146811 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.146802    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5571f5ec-1c18-4ae6-8149-47407e0b8df1" containerName="payload-pre-processing"
Sep 10 21:15:00.146966 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.146879    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5571f5ec-1c18-4ae6-8149-47407e0b8df1" containerName="payload-pre-processing"
Sep 10 21:15:00.146966 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.146891    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="360ae7ff-4219-4a46-af42-c24c9a2f961e" containerName="payload-processing"
Sep 10 21:15:00.149956 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.149931    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b22a78-29817915-tnb4z"]
Sep 10 21:15:00.150112 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.150087    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817915-thzld"
Sep 10 21:15:00.153302 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.153278    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p4gs5\""
Sep 10 21:15:00.154096 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.154075    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b22a78-29817915-tnb4z"
Sep 10 21:15:00.158392 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.157827    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817915-thzld"]
Sep 10 21:15:00.160955 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.160837    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b22a78-29817915-tnb4z"]
Sep 10 21:15:00.164291 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.164268    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817915-thzld"
Sep 10 21:15:00.173285 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.173265    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b22a78-29817915-tnb4z"
Sep 10 21:15:00.444933 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.444860    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817915-thzld"]
Sep 10 21:15:00.448422 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:15:00.448390    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e906973_36db_499f_ae89_1137beed120e.slice/crio-7dca61aa61f23f5c13775b6053f93b82b9bb1cfbd3d8033af8b62fa6da9f2c67 WatchSource:0}: Error finding container 7dca61aa61f23f5c13775b6053f93b82b9bb1cfbd3d8033af8b62fa6da9f2c67: Status 404 returned error can't find the container with id 7dca61aa61f23f5c13775b6053f93b82b9bb1cfbd3d8033af8b62fa6da9f2c67
Sep 10 21:15:00.450845 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.450828    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:15:00.457079 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.456985    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b22a78-29817915-tnb4z"]
Sep 10 21:15:00.459477 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:15:00.459454    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6102d05_b301_4d01_bf94_52c77bd80c2d.slice/crio-59ec593da817526ecd0431733ebdf2f2571a957dc66414093991d58b2dbd364a WatchSource:0}: Error finding container 59ec593da817526ecd0431733ebdf2f2571a957dc66414093991d58b2dbd364a: Status 404 returned error can't find the container with id 59ec593da817526ecd0431733ebdf2f2571a957dc66414093991d58b2dbd364a
Sep 10 21:15:00.747164 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.747049    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b22a78-29817915-tnb4z" event={"ID":"c6102d05-b301-4d01-bf94-52c77bd80c2d","Type":"ContainerStarted","Data":"59ec593da817526ecd0431733ebdf2f2571a957dc66414093991d58b2dbd364a"}
Sep 10 21:15:00.748470 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:00.748441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817915-thzld" event={"ID":"4e906973-36db-499f-ae89-1137beed120e","Type":"ContainerStarted","Data":"7dca61aa61f23f5c13775b6053f93b82b9bb1cfbd3d8033af8b62fa6da9f2c67"}
Sep 10 21:15:03.768598 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:03.768511    2580 generic.go:356] "Generic (PLEG): container finished" podID="c6102d05-b301-4d01-bf94-52c77bd80c2d" containerID="af9e29650ebd23dd409d88b74363ffd36ef4acc48b01b2945a58dcc8e08e7806" exitCode=0
Sep 10 21:15:03.768980 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:03.768602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b22a78-29817915-tnb4z" event={"ID":"c6102d05-b301-4d01-bf94-52c77bd80c2d","Type":"ContainerDied","Data":"af9e29650ebd23dd409d88b74363ffd36ef4acc48b01b2945a58dcc8e08e7806"}
Sep 10 21:15:03.770204 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:03.770181    2580 generic.go:356] "Generic (PLEG): container finished" podID="4e906973-36db-499f-ae89-1137beed120e" containerID="bab6f3782318878f5e0223315741d7b25b4f4ca31abbde44d6cca929d9a051e0" exitCode=0
Sep 10 21:15:03.770313 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:03.770210    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817915-thzld" event={"ID":"4e906973-36db-499f-ae89-1137beed120e","Type":"ContainerDied","Data":"bab6f3782318878f5e0223315741d7b25b4f4ca31abbde44d6cca929d9a051e0"}
Sep 10 21:15:03.953320 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:03.953284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:04.906311 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:04.906284    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b22a78-29817915-tnb4z"
Sep 10 21:15:04.939982 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:04.939950    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817915-thzld"
Sep 10 21:15:05.782283 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.782246    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b22a78-29817915-tnb4z"
Sep 10 21:15:05.782283 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.782264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b22a78-29817915-tnb4z" event={"ID":"c6102d05-b301-4d01-bf94-52c77bd80c2d","Type":"ContainerDied","Data":"59ec593da817526ecd0431733ebdf2f2571a957dc66414093991d58b2dbd364a"}
Sep 10 21:15:05.782525 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.782312    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59ec593da817526ecd0431733ebdf2f2571a957dc66414093991d58b2dbd364a"
Sep 10 21:15:05.783567 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.783542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817915-thzld" event={"ID":"4e906973-36db-499f-ae89-1137beed120e","Type":"ContainerDied","Data":"7dca61aa61f23f5c13775b6053f93b82b9bb1cfbd3d8033af8b62fa6da9f2c67"}
Sep 10 21:15:05.783680 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.783579    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7dca61aa61f23f5c13775b6053f93b82b9bb1cfbd3d8033af8b62fa6da9f2c67"
Sep 10 21:15:05.783680 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.783546    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817915-thzld"
Sep 10 21:15:05.865051 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.865003    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:05.987627 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.987588    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg"]
Sep 10 21:15:05.988282 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.988262    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6102d05-b301-4d01-bf94-52c77bd80c2d" containerName="cleanup"
Sep 10 21:15:05.988339 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.988285    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6102d05-b301-4d01-bf94-52c77bd80c2d" containerName="cleanup"
Sep 10 21:15:05.988339 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.988298    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e906973-36db-499f-ae89-1137beed120e" containerName="cleanup"
Sep 10 21:15:05.988339 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.988307    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e906973-36db-499f-ae89-1137beed120e" containerName="cleanup"
Sep 10 21:15:05.988434 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.988421    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e906973-36db-499f-ae89-1137beed120e" containerName="cleanup"
Sep 10 21:15:05.988469 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.988434    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6102d05-b301-4d01-bf94-52c77bd80c2d" containerName="cleanup"
Sep 10 21:15:05.991992 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.991971    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg"
Sep 10 21:15:05.994700 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.994679    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:15:05.994834 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.994817    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p4gs5\""
Sep 10 21:15:05.999274 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:05.999175    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg"]
Sep 10 21:15:06.046392 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:06.046358    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6da536b-58dd-41ca-a33d-4d6fb217f8af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg\" (UID: \"f6da536b-58dd-41ca-a33d-4d6fb217f8af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg"
Sep 10 21:15:06.147587 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:06.147548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6da536b-58dd-41ca-a33d-4d6fb217f8af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg\" (UID: \"f6da536b-58dd-41ca-a33d-4d6fb217f8af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg"
Sep 10 21:15:06.148295 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:06.148270    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6da536b-58dd-41ca-a33d-4d6fb217f8af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg\" (UID: \"f6da536b-58dd-41ca-a33d-4d6fb217f8af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg"
Sep 10 21:15:06.183601 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:06.183567    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:06.306786 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:06.306690    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg"
Sep 10 21:15:06.578555 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:06.578529    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg"]
Sep 10 21:15:06.580766 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:15:06.580737    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6da536b_58dd_41ca_a33d_4d6fb217f8af.slice/crio-47bc9186529321c2428d0aa54944a8f9dba64d69df84a9e0781361600a4fda02 WatchSource:0}: Error finding container 47bc9186529321c2428d0aa54944a8f9dba64d69df84a9e0781361600a4fda02: Status 404 returned error can't find the container with id 47bc9186529321c2428d0aa54944a8f9dba64d69df84a9e0781361600a4fda02
Sep 10 21:15:06.790870 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:06.790833    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg" event={"ID":"f6da536b-58dd-41ca-a33d-4d6fb217f8af","Type":"ContainerStarted","Data":"47bc9186529321c2428d0aa54944a8f9dba64d69df84a9e0781361600a4fda02"}
Sep 10 21:15:07.539036 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:07.538997    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:07.797073 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:07.797030    2580 generic.go:356] "Generic (PLEG): container finished" podID="f6da536b-58dd-41ca-a33d-4d6fb217f8af" containerID="d3877def28ce8e97b8c751c58e4c7929d31a412489326ec7d9de2190b1ac8960" exitCode=0
Sep 10 21:15:07.797262 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:07.797111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg" event={"ID":"f6da536b-58dd-41ca-a33d-4d6fb217f8af","Type":"ContainerDied","Data":"d3877def28ce8e97b8c751c58e4c7929d31a412489326ec7d9de2190b1ac8960"}
Sep 10 21:15:08.935767 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:08.935741    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg"
Sep 10 21:15:09.069192 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:09.069103    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6da536b-58dd-41ca-a33d-4d6fb217f8af-maas-api-service-ca\") pod \"f6da536b-58dd-41ca-a33d-4d6fb217f8af\" (UID: \"f6da536b-58dd-41ca-a33d-4d6fb217f8af\") "
Sep 10 21:15:09.069461 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:09.069437    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6da536b-58dd-41ca-a33d-4d6fb217f8af-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f6da536b-58dd-41ca-a33d-4d6fb217f8af" (UID: "f6da536b-58dd-41ca-a33d-4d6fb217f8af"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:15:09.169675 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:09.169641    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6da536b-58dd-41ca-a33d-4d6fb217f8af-maas-api-service-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:15:09.634772 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:09.634731    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:15:09.808389 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:09.808359    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg"
Sep 10 21:15:09.808558 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:09.808402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg" event={"ID":"f6da536b-58dd-41ca-a33d-4d6fb217f8af","Type":"ContainerDied","Data":"47bc9186529321c2428d0aa54944a8f9dba64d69df84a9e0781361600a4fda02"}
Sep 10 21:15:09.808558 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:09.808446    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47bc9186529321c2428d0aa54944a8f9dba64d69df84a9e0781361600a4fda02"
Sep 10 21:15:11.083092 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:11.083039    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:11.182121 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:11.182085    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:13.752138 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:13.752050    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:13.811103 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:13.811038    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:13.883505 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:13.883471    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:13.993715 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:13.993678    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:16.616776 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:16.616736    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:16.655210 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:16.655172    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"]
Sep 10 21:15:16.655495 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:16.655445    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd" podUID="832cd765-2f27-4a69-a9a2-18581ae6a084" containerName="payload-processing" containerID="cri-o://70057923e17c514a1083ed1251771e6963f70013b77347712a050861d6f9a5a6" gracePeriod=30
Sep 10 21:15:16.714935 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:16.714900    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:16.810441 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:16.810401    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg"]
Sep 10 21:15:16.812548 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:16.812514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:16.813072 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:16.813034    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-2bbc78-g49jg"]
Sep 10 21:15:16.848768 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:16.848731    2580 generic.go:356] "Generic (PLEG): container finished" podID="832cd765-2f27-4a69-a9a2-18581ae6a084" containerID="70057923e17c514a1083ed1251771e6963f70013b77347712a050861d6f9a5a6" exitCode=0
Sep 10 21:15:16.849073 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:16.848783    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd" event={"ID":"832cd765-2f27-4a69-a9a2-18581ae6a084","Type":"ContainerDied","Data":"70057923e17c514a1083ed1251771e6963f70013b77347712a050861d6f9a5a6"}
Sep 10 21:15:16.872892 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:16.872812    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:16.933006 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:16.932981    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"
Sep 10 21:15:16.958597 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:16.958567    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6da536b-58dd-41ca-a33d-4d6fb217f8af" path="/var/lib/kubelet/pods/f6da536b-58dd-41ca-a33d-4d6fb217f8af/volumes"
Sep 10 21:15:17.035859 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:17.035817    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/832cd765-2f27-4a69-a9a2-18581ae6a084-plugins-config-volume\") pod \"832cd765-2f27-4a69-a9a2-18581ae6a084\" (UID: \"832cd765-2f27-4a69-a9a2-18581ae6a084\") "
Sep 10 21:15:17.036033 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:17.035879    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpw96\" (UniqueName: \"kubernetes.io/projected/832cd765-2f27-4a69-a9a2-18581ae6a084-kube-api-access-kpw96\") pod \"832cd765-2f27-4a69-a9a2-18581ae6a084\" (UID: \"832cd765-2f27-4a69-a9a2-18581ae6a084\") "
Sep 10 21:15:17.036426 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:17.036292    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/832cd765-2f27-4a69-a9a2-18581ae6a084-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "832cd765-2f27-4a69-a9a2-18581ae6a084" (UID: "832cd765-2f27-4a69-a9a2-18581ae6a084"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:15:17.037994 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:17.037969    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/832cd765-2f27-4a69-a9a2-18581ae6a084-kube-api-access-kpw96" (OuterVolumeSpecName: "kube-api-access-kpw96") pod "832cd765-2f27-4a69-a9a2-18581ae6a084" (UID: "832cd765-2f27-4a69-a9a2-18581ae6a084"). InnerVolumeSpecName "kube-api-access-kpw96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:15:17.137049 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:17.136958    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/832cd765-2f27-4a69-a9a2-18581ae6a084-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:15:17.137049 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:17.136991    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpw96\" (UniqueName: \"kubernetes.io/projected/832cd765-2f27-4a69-a9a2-18581ae6a084-kube-api-access-kpw96\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:15:17.203507 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:17.203471    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:17.862850 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:17.862817    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"
Sep 10 21:15:17.863378 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:17.862846    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd" event={"ID":"832cd765-2f27-4a69-a9a2-18581ae6a084","Type":"ContainerDied","Data":"5f64ff99d0325574bd4c11436b34590bba33b74a7cab35fa20c0a9c0d35fd528"}
Sep 10 21:15:17.863378 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:17.862925    2580 scope.go:117] "RemoveContainer" containerID="70057923e17c514a1083ed1251771e6963f70013b77347712a050861d6f9a5a6"
Sep 10 21:15:17.887091 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:17.887042    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"]
Sep 10 21:15:17.890782 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:17.890755    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-2bbc78-54c5d84bf-rsfjd"]
Sep 10 21:15:18.956451 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:18.956410    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="832cd765-2f27-4a69-a9a2-18581ae6a084" path="/var/lib/kubelet/pods/832cd765-2f27-4a69-a9a2-18581ae6a084/volumes"
Sep 10 21:15:23.039310 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:23.039276    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:15:23.544069 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:23.544011    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:23.644839 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:23.644807    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:23.831465 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:23.831386    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:24.288039 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:24.288000    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:24.610946 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:24.610811    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:24.681439 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:24.681403    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:26.089304 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:26.089268    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:26.188136 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:26.188105    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:26.308039 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:26.307987    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:29.848722 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:29.848689    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:35.721050 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:35.721013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:35.825545 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:35.825512    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:35.921978 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:35.921950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:36.026591 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:36.026508    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:46.456241 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:46.456207    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:47.283506 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:47.283473    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:50.796831 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:50.796795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:53.519841 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:53.519803    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:53.580265 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:53.580239    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:53.670927 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:53.670900    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:53.765665 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:53.765641    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:53.867356 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:53.867332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:54.473390 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:54.473354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:54.570085 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:54.570037    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:54.670325 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:54.670297    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:56.199804 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:56.199770    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:56.725711 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:56.725676    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:56.892454 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:56.892406    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:57.092476 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:57.092439    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:57.216820 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:57.216782    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:15:59.639170 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:15:59.639136    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:16:02.393016 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:02.392980    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:02.454563 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:02.454532    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:03.017980 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:03.017946    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:06.153852 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:06.153816    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:08.794500 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:08.794468    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:09.819358 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:09.819294    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:09.931850 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:09.931814    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj"]
Sep 10 21:16:09.932262 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:09.932248    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6da536b-58dd-41ca-a33d-4d6fb217f8af" containerName="revoke-keys"
Sep 10 21:16:09.932314 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:09.932263    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6da536b-58dd-41ca-a33d-4d6fb217f8af" containerName="revoke-keys"
Sep 10 21:16:09.932314 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:09.932283    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="832cd765-2f27-4a69-a9a2-18581ae6a084" containerName="payload-processing"
Sep 10 21:16:09.932314 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:09.932289    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="832cd765-2f27-4a69-a9a2-18581ae6a084" containerName="payload-processing"
Sep 10 21:16:09.932413 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:09.932347    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6da536b-58dd-41ca-a33d-4d6fb217f8af" containerName="revoke-keys"
Sep 10 21:16:09.932413 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:09.932358    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="832cd765-2f27-4a69-a9a2-18581ae6a084" containerName="payload-processing"
Sep 10 21:16:09.935560 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:09.935545    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj"
Sep 10 21:16:09.938295 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:09.938269    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p4gs5\""
Sep 10 21:16:09.938431 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:09.938309    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:16:09.943950 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:09.943929    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj"]
Sep 10 21:16:09.991363 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:09.991320    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d67523be-197b-4faf-b835-537e6f47a4eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj\" (UID: \"d67523be-197b-4faf-b835-537e6f47a4eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj"
Sep 10 21:16:10.059109 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:10.059045    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:10.092329 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:10.092242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d67523be-197b-4faf-b835-537e6f47a4eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj\" (UID: \"d67523be-197b-4faf-b835-537e6f47a4eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj"
Sep 10 21:16:10.092876 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:10.092857    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d67523be-197b-4faf-b835-537e6f47a4eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj\" (UID: \"d67523be-197b-4faf-b835-537e6f47a4eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj"
Sep 10 21:16:10.247328 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:10.247292    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj"
Sep 10 21:16:10.256111 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:10.256048    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:10.358154 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:10.358044    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:10.485285 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:10.485250    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:10.519965 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:10.519938    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj"]
Sep 10 21:16:10.521786 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:16:10.521752    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd67523be_197b_4faf_b835_537e6f47a4eb.slice/crio-e43cb60670c14932e2ae3e45043e1dd1fd9e3de2640f5ecced1200f303cb4f1d WatchSource:0}: Error finding container e43cb60670c14932e2ae3e45043e1dd1fd9e3de2640f5ecced1200f303cb4f1d: Status 404 returned error can't find the container with id e43cb60670c14932e2ae3e45043e1dd1fd9e3de2640f5ecced1200f303cb4f1d
Sep 10 21:16:10.810121 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:10.810090    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:11.100724 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:11.100689    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:11.142776 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:11.142739    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj" event={"ID":"d67523be-197b-4faf-b835-537e6f47a4eb","Type":"ContainerStarted","Data":"1d73fbffc647f02b1dbf14079d79db6e826caf0f943d71b14d706cd6ad897e2c"}
Sep 10 21:16:11.142963 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:11.142797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj" event={"ID":"d67523be-197b-4faf-b835-537e6f47a4eb","Type":"ContainerStarted","Data":"e43cb60670c14932e2ae3e45043e1dd1fd9e3de2640f5ecced1200f303cb4f1d"}
Sep 10 21:16:11.160321 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:11.160267    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj" podStartSLOduration=2.160251413 podStartE2EDuration="2.160251413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:11.158028073 +0000 UTC m=+1064.837526925" watchObservedRunningTime="2026-09-10 21:16:11.160251413 +0000 UTC m=+1064.839750263"
Sep 10 21:16:12.149747 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:12.149711    2580 generic.go:356] "Generic (PLEG): container finished" podID="d67523be-197b-4faf-b835-537e6f47a4eb" containerID="1d73fbffc647f02b1dbf14079d79db6e826caf0f943d71b14d706cd6ad897e2c" exitCode=0
Sep 10 21:16:12.150164 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:12.149789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj" event={"ID":"d67523be-197b-4faf-b835-537e6f47a4eb","Type":"ContainerDied","Data":"1d73fbffc647f02b1dbf14079d79db6e826caf0f943d71b14d706cd6ad897e2c"}
Sep 10 21:16:12.843092 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:12.843037    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:16:13.292750 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:13.292724    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj"
Sep 10 21:16:13.322160 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:13.322127    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d67523be-197b-4faf-b835-537e6f47a4eb-maas-api-service-ca\") pod \"d67523be-197b-4faf-b835-537e6f47a4eb\" (UID: \"d67523be-197b-4faf-b835-537e6f47a4eb\") "
Sep 10 21:16:13.322513 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:13.322489    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d67523be-197b-4faf-b835-537e6f47a4eb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d67523be-197b-4faf-b835-537e6f47a4eb" (UID: "d67523be-197b-4faf-b835-537e6f47a4eb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:16:13.423532 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:13.423445    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d67523be-197b-4faf-b835-537e6f47a4eb-maas-api-service-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:16:14.016208 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:14.016167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:14.106829 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:14.106789    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:14.161657 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:14.161617    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj" event={"ID":"d67523be-197b-4faf-b835-537e6f47a4eb","Type":"ContainerDied","Data":"e43cb60670c14932e2ae3e45043e1dd1fd9e3de2640f5ecced1200f303cb4f1d"}
Sep 10 21:16:14.161657 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:14.161657    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj"
Sep 10 21:16:14.161868 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:14.161702    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e43cb60670c14932e2ae3e45043e1dd1fd9e3de2640f5ecced1200f303cb4f1d"
Sep 10 21:16:15.187010 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:15.186977    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:15.284818 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:15.284781    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:15.342580 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:15.342548    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:15.767230 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:15.767172    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:15.865490 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:15.865441    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:15.957281 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:15.957246    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"]
Sep 10 21:16:15.957663 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:15.957651    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d67523be-197b-4faf-b835-537e6f47a4eb" containerName="revoke-keys"
Sep 10 21:16:15.957720 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:15.957665    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d67523be-197b-4faf-b835-537e6f47a4eb" containerName="revoke-keys"
Sep 10 21:16:15.957756 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:15.957751    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d67523be-197b-4faf-b835-537e6f47a4eb" containerName="revoke-keys"
Sep 10 21:16:15.960984 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:15.960966    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"
Sep 10 21:16:15.963596 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:15.963576    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-170a718d\""
Sep 10 21:16:15.963596 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:15.963596    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-170a718d-dockercfg-g66zv\""
Sep 10 21:16:15.972393 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:15.972357    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"]
Sep 10 21:16:16.048253 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:16.048226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d95p5\" (UniqueName: \"kubernetes.io/projected/a9d28fcf-f470-4092-9e7c-45bcfb7ea95c-kube-api-access-d95p5\") pod \"payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c\" (UID: \"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"
Sep 10 21:16:16.048429 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:16.048262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9d28fcf-f470-4092-9e7c-45bcfb7ea95c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c\" (UID: \"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"
Sep 10 21:16:16.148799 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:16.148761    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d95p5\" (UniqueName: \"kubernetes.io/projected/a9d28fcf-f470-4092-9e7c-45bcfb7ea95c-kube-api-access-d95p5\") pod \"payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c\" (UID: \"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"
Sep 10 21:16:16.148934 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:16.148806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9d28fcf-f470-4092-9e7c-45bcfb7ea95c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c\" (UID: \"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"
Sep 10 21:16:16.149583 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:16.149564    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9d28fcf-f470-4092-9e7c-45bcfb7ea95c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c\" (UID: \"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"
Sep 10 21:16:16.158348 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:16.158314    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d95p5\" (UniqueName: \"kubernetes.io/projected/a9d28fcf-f470-4092-9e7c-45bcfb7ea95c-kube-api-access-d95p5\") pod \"payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c\" (UID: \"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"
Sep 10 21:16:16.272497 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:16.272461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"
Sep 10 21:16:16.547227 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:16.547027    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"]
Sep 10 21:16:16.549931 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:16:16.549897    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9d28fcf_f470_4092_9e7c_45bcfb7ea95c.slice/crio-a22707aa012dde3ec4c7793a78e8fc18af268b9056aed5dae3ea8d6b8d02be82 WatchSource:0}: Error finding container a22707aa012dde3ec4c7793a78e8fc18af268b9056aed5dae3ea8d6b8d02be82: Status 404 returned error can't find the container with id a22707aa012dde3ec4c7793a78e8fc18af268b9056aed5dae3ea8d6b8d02be82
Sep 10 21:16:17.179706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:17.179665    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c" event={"ID":"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c","Type":"ContainerStarted","Data":"d78358612132093334b91200893328dfbe61cbcf4f2a2b999128a37a2e98f43e"}
Sep 10 21:16:17.179890 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:17.179724    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c" event={"ID":"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c","Type":"ContainerStarted","Data":"a22707aa012dde3ec4c7793a78e8fc18af268b9056aed5dae3ea8d6b8d02be82"}
Sep 10 21:16:17.179890 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:17.179754    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"
Sep 10 21:16:17.201576 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:17.201517    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c" podStartSLOduration=2.201497392 podStartE2EDuration="2.201497392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:17.199442378 +0000 UTC m=+1070.878941229" watchObservedRunningTime="2026-09-10 21:16:17.201497392 +0000 UTC m=+1070.880996245"
Sep 10 21:16:20.072267 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.072179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:20.720482 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.720451    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:20.737576 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.737535    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b22a78-29817915-tnb4z"]
Sep 10 21:16:20.740013 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.739986    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-b22a78-29817915-tnb4z"]
Sep 10 21:16:20.760963 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.760930    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"]
Sep 10 21:16:20.761341 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.761289    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv" podUID="1ceaf786-2b54-45f1-9e05-171db5b1feb6" containerName="payload-processing" containerID="cri-o://f80dbd28d5a0a75f9255126e0519309d18547389fee5717b79b02d6db1a216fb" gracePeriod=30
Sep 10 21:16:20.766976 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.766950    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"]
Sep 10 21:16:20.767390 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.767362    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj" podUID="e252a656-33bf-482c-aeeb-71b9b2022f48" containerName="payload-pre-processing" containerID="cri-o://e7d71109a4e1e7c94ff4dc2fbcb68410939fcb20d626f5ae059468f3cd59055d" gracePeriod=30
Sep 10 21:16:20.793379 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.793356    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:20.853848 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.853815    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:20.953285 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.953252    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:20.957278 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.957251    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6102d05-b301-4d01-bf94-52c77bd80c2d" path="/var/lib/kubelet/pods/c6102d05-b301-4d01-bf94-52c77bd80c2d/volumes"
Sep 10 21:16:20.957501 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.957488    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj"]
Sep 10 21:16:20.957565 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:20.957514    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b22a78-8p4wj"]
Sep 10 21:16:21.052486 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.052461    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"
Sep 10 21:16:21.052874 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.052851    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:21.055275 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.055255    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"
Sep 10 21:16:21.099501 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.099471    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzjrb\" (UniqueName: \"kubernetes.io/projected/e252a656-33bf-482c-aeeb-71b9b2022f48-kube-api-access-zzjrb\") pod \"e252a656-33bf-482c-aeeb-71b9b2022f48\" (UID: \"e252a656-33bf-482c-aeeb-71b9b2022f48\") "
Sep 10 21:16:21.099954 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.099516    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e252a656-33bf-482c-aeeb-71b9b2022f48-plugins-config-volume\") pod \"e252a656-33bf-482c-aeeb-71b9b2022f48\" (UID: \"e252a656-33bf-482c-aeeb-71b9b2022f48\") "
Sep 10 21:16:21.099954 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.099541    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ceaf786-2b54-45f1-9e05-171db5b1feb6-plugins-config-volume\") pod \"1ceaf786-2b54-45f1-9e05-171db5b1feb6\" (UID: \"1ceaf786-2b54-45f1-9e05-171db5b1feb6\") "
Sep 10 21:16:21.099954 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.099669    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nlsn\" (UniqueName: \"kubernetes.io/projected/1ceaf786-2b54-45f1-9e05-171db5b1feb6-kube-api-access-7nlsn\") pod \"1ceaf786-2b54-45f1-9e05-171db5b1feb6\" (UID: \"1ceaf786-2b54-45f1-9e05-171db5b1feb6\") "
Sep 10 21:16:21.099954 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.099938    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ceaf786-2b54-45f1-9e05-171db5b1feb6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1ceaf786-2b54-45f1-9e05-171db5b1feb6" (UID: "1ceaf786-2b54-45f1-9e05-171db5b1feb6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:16:21.100186 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.099938    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e252a656-33bf-482c-aeeb-71b9b2022f48-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e252a656-33bf-482c-aeeb-71b9b2022f48" (UID: "e252a656-33bf-482c-aeeb-71b9b2022f48"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:16:21.101663 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.101638    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e252a656-33bf-482c-aeeb-71b9b2022f48-kube-api-access-zzjrb" (OuterVolumeSpecName: "kube-api-access-zzjrb") pod "e252a656-33bf-482c-aeeb-71b9b2022f48" (UID: "e252a656-33bf-482c-aeeb-71b9b2022f48"). InnerVolumeSpecName "kube-api-access-zzjrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:16:21.101774 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.101733    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ceaf786-2b54-45f1-9e05-171db5b1feb6-kube-api-access-7nlsn" (OuterVolumeSpecName: "kube-api-access-7nlsn") pod "1ceaf786-2b54-45f1-9e05-171db5b1feb6" (UID: "1ceaf786-2b54-45f1-9e05-171db5b1feb6"). InnerVolumeSpecName "kube-api-access-7nlsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:16:21.200469 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.200436    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nlsn\" (UniqueName: \"kubernetes.io/projected/1ceaf786-2b54-45f1-9e05-171db5b1feb6-kube-api-access-7nlsn\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:16:21.200469 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.200469    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzjrb\" (UniqueName: \"kubernetes.io/projected/e252a656-33bf-482c-aeeb-71b9b2022f48-kube-api-access-zzjrb\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:16:21.200469 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.200480    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e252a656-33bf-482c-aeeb-71b9b2022f48-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:16:21.200753 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.200488    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ceaf786-2b54-45f1-9e05-171db5b1feb6-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:16:21.201339 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.201315    2580 generic.go:356] "Generic (PLEG): container finished" podID="e252a656-33bf-482c-aeeb-71b9b2022f48" containerID="e7d71109a4e1e7c94ff4dc2fbcb68410939fcb20d626f5ae059468f3cd59055d" exitCode=0
Sep 10 21:16:21.201431 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.201363    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"
Sep 10 21:16:21.201493 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.201447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj" event={"ID":"e252a656-33bf-482c-aeeb-71b9b2022f48","Type":"ContainerDied","Data":"e7d71109a4e1e7c94ff4dc2fbcb68410939fcb20d626f5ae059468f3cd59055d"}
Sep 10 21:16:21.201549 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.201496    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj" event={"ID":"e252a656-33bf-482c-aeeb-71b9b2022f48","Type":"ContainerDied","Data":"00030b2c7ce43768c4cb6a94458005f4c7c3e8a38255c08142086c8afd60fcce"}
Sep 10 21:16:21.201549 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.201526    2580 scope.go:117] "RemoveContainer" containerID="e7d71109a4e1e7c94ff4dc2fbcb68410939fcb20d626f5ae059468f3cd59055d"
Sep 10 21:16:21.202951 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.202925    2580 generic.go:356] "Generic (PLEG): container finished" podID="1ceaf786-2b54-45f1-9e05-171db5b1feb6" containerID="f80dbd28d5a0a75f9255126e0519309d18547389fee5717b79b02d6db1a216fb" exitCode=0
Sep 10 21:16:21.203095 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.202972    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"
Sep 10 21:16:21.203095 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.202992    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv" event={"ID":"1ceaf786-2b54-45f1-9e05-171db5b1feb6","Type":"ContainerDied","Data":"f80dbd28d5a0a75f9255126e0519309d18547389fee5717b79b02d6db1a216fb"}
Sep 10 21:16:21.203095 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.203033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv" event={"ID":"1ceaf786-2b54-45f1-9e05-171db5b1feb6","Type":"ContainerDied","Data":"74e9501b4b1139a4e7e6f42118817b302c88886038ed752afb0b2b1c0cb4be0c"}
Sep 10 21:16:21.212492 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.212477    2580 scope.go:117] "RemoveContainer" containerID="e7d71109a4e1e7c94ff4dc2fbcb68410939fcb20d626f5ae059468f3cd59055d"
Sep 10 21:16:21.212777 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:16:21.212757    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7d71109a4e1e7c94ff4dc2fbcb68410939fcb20d626f5ae059468f3cd59055d\": container with ID starting with e7d71109a4e1e7c94ff4dc2fbcb68410939fcb20d626f5ae059468f3cd59055d not found: ID does not exist" containerID="e7d71109a4e1e7c94ff4dc2fbcb68410939fcb20d626f5ae059468f3cd59055d"
Sep 10 21:16:21.212824 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.212785    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7d71109a4e1e7c94ff4dc2fbcb68410939fcb20d626f5ae059468f3cd59055d"} err="failed to get container status \"e7d71109a4e1e7c94ff4dc2fbcb68410939fcb20d626f5ae059468f3cd59055d\": rpc error: code = NotFound desc = could not find container \"e7d71109a4e1e7c94ff4dc2fbcb68410939fcb20d626f5ae059468f3cd59055d\": container with ID starting with e7d71109a4e1e7c94ff4dc2fbcb68410939fcb20d626f5ae059468f3cd59055d not found: ID does not exist"
Sep 10 21:16:21.212824 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.212802    2580 scope.go:117] "RemoveContainer" containerID="f80dbd28d5a0a75f9255126e0519309d18547389fee5717b79b02d6db1a216fb"
Sep 10 21:16:21.221188 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.221167    2580 scope.go:117] "RemoveContainer" containerID="f80dbd28d5a0a75f9255126e0519309d18547389fee5717b79b02d6db1a216fb"
Sep 10 21:16:21.221467 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:16:21.221424    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f80dbd28d5a0a75f9255126e0519309d18547389fee5717b79b02d6db1a216fb\": container with ID starting with f80dbd28d5a0a75f9255126e0519309d18547389fee5717b79b02d6db1a216fb not found: ID does not exist" containerID="f80dbd28d5a0a75f9255126e0519309d18547389fee5717b79b02d6db1a216fb"
Sep 10 21:16:21.221467 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.221452    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f80dbd28d5a0a75f9255126e0519309d18547389fee5717b79b02d6db1a216fb"} err="failed to get container status \"f80dbd28d5a0a75f9255126e0519309d18547389fee5717b79b02d6db1a216fb\": rpc error: code = NotFound desc = could not find container \"f80dbd28d5a0a75f9255126e0519309d18547389fee5717b79b02d6db1a216fb\": container with ID starting with f80dbd28d5a0a75f9255126e0519309d18547389fee5717b79b02d6db1a216fb not found: ID does not exist"
Sep 10 21:16:21.225223 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.225201    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"]
Sep 10 21:16:21.229445 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.229423    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b22a78-7d55745798-48tmj"]
Sep 10 21:16:21.240694 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.240669    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"]
Sep 10 21:16:21.244598 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.244576    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b22a78-57c4949d5-kx7fv"]
Sep 10 21:16:21.324737 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:21.324708    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:22.957922 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:22.957888    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ceaf786-2b54-45f1-9e05-171db5b1feb6" path="/var/lib/kubelet/pods/1ceaf786-2b54-45f1-9e05-171db5b1feb6/volumes"
Sep 10 21:16:22.958317 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:22.958239    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d67523be-197b-4faf-b835-537e6f47a4eb" path="/var/lib/kubelet/pods/d67523be-197b-4faf-b835-537e6f47a4eb/volumes"
Sep 10 21:16:22.958561 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:22.958548    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e252a656-33bf-482c-aeeb-71b9b2022f48" path="/var/lib/kubelet/pods/e252a656-33bf-482c-aeeb-71b9b2022f48/volumes"
Sep 10 21:16:24.942207 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:24.942170    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:16:27.635845 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:27.635806    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:27.727468 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:27.727442    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:27.934883 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:27.934810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:28.187296 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:28.187209    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"
Sep 10 21:16:28.533500 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:28.533425    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:37.256894 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:37.256862    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:38.886419 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:38.886387    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:38.921107 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:38.921044    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"]
Sep 10 21:16:38.921351 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:38.921322    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7" podUID="659e7b52-25bc-4f02-850b-cda467acfc01" containerName="payload-processing" containerID="cri-o://3059028c6b7fcbd9c35cf43c7462481c1141b16aefdb923503d9822bf62e9c67" gracePeriod=30
Sep 10 21:16:38.926366 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:38.926339    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"]
Sep 10 21:16:38.927032 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:38.927003    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5" podUID="77a14455-f664-4f69-b08b-63efb0aa8898" containerName="payload-pre-processing" containerID="cri-o://bfc56ab4d9fbe3ba979ae4ced1fafa54233f21c4cf362968b220c9c1278d9562" gracePeriod=30
Sep 10 21:16:38.947897 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:38.947864    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:39.010917 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.010886    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:39.084600 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.084569    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:39.143845 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.143769    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:39.203223 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.203200    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"
Sep 10 21:16:39.216578 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.216553    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"
Sep 10 21:16:39.309994 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.309961    2580 generic.go:356] "Generic (PLEG): container finished" podID="77a14455-f664-4f69-b08b-63efb0aa8898" containerID="bfc56ab4d9fbe3ba979ae4ced1fafa54233f21c4cf362968b220c9c1278d9562" exitCode=0
Sep 10 21:16:39.309994 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.309993    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"
Sep 10 21:16:39.310264 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.310031    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5" event={"ID":"77a14455-f664-4f69-b08b-63efb0aa8898","Type":"ContainerDied","Data":"bfc56ab4d9fbe3ba979ae4ced1fafa54233f21c4cf362968b220c9c1278d9562"}
Sep 10 21:16:39.310264 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.310119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5" event={"ID":"77a14455-f664-4f69-b08b-63efb0aa8898","Type":"ContainerDied","Data":"8242e0a43bfd2b5c4db9edfe2bbbe87ee36ddf565bd5353f38790847ce110699"}
Sep 10 21:16:39.310264 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.310150    2580 scope.go:117] "RemoveContainer" containerID="bfc56ab4d9fbe3ba979ae4ced1fafa54233f21c4cf362968b220c9c1278d9562"
Sep 10 21:16:39.311504 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.311480    2580 generic.go:356] "Generic (PLEG): container finished" podID="659e7b52-25bc-4f02-850b-cda467acfc01" containerID="3059028c6b7fcbd9c35cf43c7462481c1141b16aefdb923503d9822bf62e9c67" exitCode=0
Sep 10 21:16:39.311628 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.311506    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7" event={"ID":"659e7b52-25bc-4f02-850b-cda467acfc01","Type":"ContainerDied","Data":"3059028c6b7fcbd9c35cf43c7462481c1141b16aefdb923503d9822bf62e9c67"}
Sep 10 21:16:39.311628 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.311529    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"
Sep 10 21:16:39.311628 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.311552    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7" event={"ID":"659e7b52-25bc-4f02-850b-cda467acfc01","Type":"ContainerDied","Data":"81929d0f481acd91a1d782c5e1fc9e9d5d9d4e373edca3701a2f7607cd8ac20d"}
Sep 10 21:16:39.319224 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.319210    2580 scope.go:117] "RemoveContainer" containerID="bfc56ab4d9fbe3ba979ae4ced1fafa54233f21c4cf362968b220c9c1278d9562"
Sep 10 21:16:39.319494 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:16:39.319477    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfc56ab4d9fbe3ba979ae4ced1fafa54233f21c4cf362968b220c9c1278d9562\": container with ID starting with bfc56ab4d9fbe3ba979ae4ced1fafa54233f21c4cf362968b220c9c1278d9562 not found: ID does not exist" containerID="bfc56ab4d9fbe3ba979ae4ced1fafa54233f21c4cf362968b220c9c1278d9562"
Sep 10 21:16:39.319566 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.319505    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfc56ab4d9fbe3ba979ae4ced1fafa54233f21c4cf362968b220c9c1278d9562"} err="failed to get container status \"bfc56ab4d9fbe3ba979ae4ced1fafa54233f21c4cf362968b220c9c1278d9562\": rpc error: code = NotFound desc = could not find container \"bfc56ab4d9fbe3ba979ae4ced1fafa54233f21c4cf362968b220c9c1278d9562\": container with ID starting with bfc56ab4d9fbe3ba979ae4ced1fafa54233f21c4cf362968b220c9c1278d9562 not found: ID does not exist"
Sep 10 21:16:39.319566 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.319528    2580 scope.go:117] "RemoveContainer" containerID="3059028c6b7fcbd9c35cf43c7462481c1141b16aefdb923503d9822bf62e9c67"
Sep 10 21:16:39.327242 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.327221    2580 scope.go:117] "RemoveContainer" containerID="3059028c6b7fcbd9c35cf43c7462481c1141b16aefdb923503d9822bf62e9c67"
Sep 10 21:16:39.327505 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:16:39.327480    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3059028c6b7fcbd9c35cf43c7462481c1141b16aefdb923503d9822bf62e9c67\": container with ID starting with 3059028c6b7fcbd9c35cf43c7462481c1141b16aefdb923503d9822bf62e9c67 not found: ID does not exist" containerID="3059028c6b7fcbd9c35cf43c7462481c1141b16aefdb923503d9822bf62e9c67"
Sep 10 21:16:39.327586 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.327516    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3059028c6b7fcbd9c35cf43c7462481c1141b16aefdb923503d9822bf62e9c67"} err="failed to get container status \"3059028c6b7fcbd9c35cf43c7462481c1141b16aefdb923503d9822bf62e9c67\": rpc error: code = NotFound desc = could not find container \"3059028c6b7fcbd9c35cf43c7462481c1141b16aefdb923503d9822bf62e9c67\": container with ID starting with 3059028c6b7fcbd9c35cf43c7462481c1141b16aefdb923503d9822bf62e9c67 not found: ID does not exist"
Sep 10 21:16:39.362927 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.362894    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/659e7b52-25bc-4f02-850b-cda467acfc01-plugins-config-volume\") pod \"659e7b52-25bc-4f02-850b-cda467acfc01\" (UID: \"659e7b52-25bc-4f02-850b-cda467acfc01\") "
Sep 10 21:16:39.363099 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.362947    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbrm9\" (UniqueName: \"kubernetes.io/projected/77a14455-f664-4f69-b08b-63efb0aa8898-kube-api-access-wbrm9\") pod \"77a14455-f664-4f69-b08b-63efb0aa8898\" (UID: \"77a14455-f664-4f69-b08b-63efb0aa8898\") "
Sep 10 21:16:39.363099 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.362981    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77a14455-f664-4f69-b08b-63efb0aa8898-plugins-config-volume\") pod \"77a14455-f664-4f69-b08b-63efb0aa8898\" (UID: \"77a14455-f664-4f69-b08b-63efb0aa8898\") "
Sep 10 21:16:39.363099 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.363039    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6j2h\" (UniqueName: \"kubernetes.io/projected/659e7b52-25bc-4f02-850b-cda467acfc01-kube-api-access-n6j2h\") pod \"659e7b52-25bc-4f02-850b-cda467acfc01\" (UID: \"659e7b52-25bc-4f02-850b-cda467acfc01\") "
Sep 10 21:16:39.363331 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.363303    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/659e7b52-25bc-4f02-850b-cda467acfc01-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "659e7b52-25bc-4f02-850b-cda467acfc01" (UID: "659e7b52-25bc-4f02-850b-cda467acfc01"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:16:39.363386 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.363352    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77a14455-f664-4f69-b08b-63efb0aa8898-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "77a14455-f664-4f69-b08b-63efb0aa8898" (UID: "77a14455-f664-4f69-b08b-63efb0aa8898"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:16:39.365022 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.365000    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77a14455-f664-4f69-b08b-63efb0aa8898-kube-api-access-wbrm9" (OuterVolumeSpecName: "kube-api-access-wbrm9") pod "77a14455-f664-4f69-b08b-63efb0aa8898" (UID: "77a14455-f664-4f69-b08b-63efb0aa8898"). InnerVolumeSpecName "kube-api-access-wbrm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:16:39.365107 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.365034    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/659e7b52-25bc-4f02-850b-cda467acfc01-kube-api-access-n6j2h" (OuterVolumeSpecName: "kube-api-access-n6j2h") pod "659e7b52-25bc-4f02-850b-cda467acfc01" (UID: "659e7b52-25bc-4f02-850b-cda467acfc01"). InnerVolumeSpecName "kube-api-access-n6j2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:16:39.464535 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.464438    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/659e7b52-25bc-4f02-850b-cda467acfc01-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:16:39.464535 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.464469    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbrm9\" (UniqueName: \"kubernetes.io/projected/77a14455-f664-4f69-b08b-63efb0aa8898-kube-api-access-wbrm9\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:16:39.464535 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.464484    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77a14455-f664-4f69-b08b-63efb0aa8898-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:16:39.464535 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.464497    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6j2h\" (UniqueName: \"kubernetes.io/projected/659e7b52-25bc-4f02-850b-cda467acfc01-kube-api-access-n6j2h\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:16:39.482166 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.482130    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:39.647411 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.647377    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"]
Sep 10 21:16:39.659377 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.659349    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-414d5f-5ccfdfb779-x8zz7"]
Sep 10 21:16:39.676600 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.676572    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"]
Sep 10 21:16:39.680839 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.680813    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-414d5f-5599594794-6jjf5"]
Sep 10 21:16:39.769874 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.769788    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:39.863510 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:39.863476    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:40.221243 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:40.221207    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:40.404208 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:40.404176    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:40.956792 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:40.956757    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="659e7b52-25bc-4f02-850b-cda467acfc01" path="/var/lib/kubelet/pods/659e7b52-25bc-4f02-850b-cda467acfc01/volumes"
Sep 10 21:16:40.957157 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:40.957144    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77a14455-f664-4f69-b08b-63efb0aa8898" path="/var/lib/kubelet/pods/77a14455-f664-4f69-b08b-63efb0aa8898/volumes"
Sep 10 21:16:41.644220 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:41.644183    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:16:45.816924 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:45.816890    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:45.917201 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:45.917169    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:46.094727 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.094645    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:46.549455 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.549423    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:46.604085 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.604029    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:46.674860 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.674825    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:46.940417 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.940332    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv"]
Sep 10 21:16:46.940772 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.940708    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e252a656-33bf-482c-aeeb-71b9b2022f48" containerName="payload-pre-processing"
Sep 10 21:16:46.940772 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.940721    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e252a656-33bf-482c-aeeb-71b9b2022f48" containerName="payload-pre-processing"
Sep 10 21:16:46.940772 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.940735    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a14455-f664-4f69-b08b-63efb0aa8898" containerName="payload-pre-processing"
Sep 10 21:16:46.940772 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.940741    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a14455-f664-4f69-b08b-63efb0aa8898" containerName="payload-pre-processing"
Sep 10 21:16:46.940772 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.940747    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="659e7b52-25bc-4f02-850b-cda467acfc01" containerName="payload-processing"
Sep 10 21:16:46.940772 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.940753    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="659e7b52-25bc-4f02-850b-cda467acfc01" containerName="payload-processing"
Sep 10 21:16:46.940772 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.940766    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ceaf786-2b54-45f1-9e05-171db5b1feb6" containerName="payload-processing"
Sep 10 21:16:46.940772 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.940771    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ceaf786-2b54-45f1-9e05-171db5b1feb6" containerName="payload-processing"
Sep 10 21:16:46.941041 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.940851    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ceaf786-2b54-45f1-9e05-171db5b1feb6" containerName="payload-processing"
Sep 10 21:16:46.941041 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.940859    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="659e7b52-25bc-4f02-850b-cda467acfc01" containerName="payload-processing"
Sep 10 21:16:46.941041 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.940867    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e252a656-33bf-482c-aeeb-71b9b2022f48" containerName="payload-pre-processing"
Sep 10 21:16:46.941041 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.940873    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77a14455-f664-4f69-b08b-63efb0aa8898" containerName="payload-pre-processing"
Sep 10 21:16:46.945589 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.945572    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv"
Sep 10 21:16:46.948329 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.948308    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:16:46.948454 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.948436    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p4gs5\""
Sep 10 21:16:46.966178 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:46.966149    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv"]
Sep 10 21:16:47.005583 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:47.005542    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:47.034891 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:47.034841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccd5debd-9a05-4b64-bf4e-088cd1a600d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-170a718d-6gplv\" (UID: \"ccd5debd-9a05-4b64-bf4e-088cd1a600d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv"
Sep 10 21:16:47.135572 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:47.135537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccd5debd-9a05-4b64-bf4e-088cd1a600d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-170a718d-6gplv\" (UID: \"ccd5debd-9a05-4b64-bf4e-088cd1a600d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv"
Sep 10 21:16:47.136254 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:47.136230    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccd5debd-9a05-4b64-bf4e-088cd1a600d0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-170a718d-6gplv\" (UID: \"ccd5debd-9a05-4b64-bf4e-088cd1a600d0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv"
Sep 10 21:16:47.260250 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:47.260161    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv"
Sep 10 21:16:47.539770 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:47.539747    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv"]
Sep 10 21:16:47.543251 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:16:47.543219    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccd5debd_9a05_4b64_bf4e_088cd1a600d0.slice/crio-b3dac88c5563a3b4a70345b1face75121ebc12cd6a2b839638a7a313a73d898c WatchSource:0}: Error finding container b3dac88c5563a3b4a70345b1face75121ebc12cd6a2b839638a7a313a73d898c: Status 404 returned error can't find the container with id b3dac88c5563a3b4a70345b1face75121ebc12cd6a2b839638a7a313a73d898c
Sep 10 21:16:47.593405 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:47.593374    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:48.362009 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:48.361974    2580 generic.go:356] "Generic (PLEG): container finished" podID="ccd5debd-9a05-4b64-bf4e-088cd1a600d0" containerID="106bfa3b795c6f9f2197a14968851103708d7912a57d8eac4e255f059fe08579" exitCode=0
Sep 10 21:16:48.362455 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:48.362018    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv" event={"ID":"ccd5debd-9a05-4b64-bf4e-088cd1a600d0","Type":"ContainerDied","Data":"106bfa3b795c6f9f2197a14968851103708d7912a57d8eac4e255f059fe08579"}
Sep 10 21:16:48.362455 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:48.362110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv" event={"ID":"ccd5debd-9a05-4b64-bf4e-088cd1a600d0","Type":"ContainerStarted","Data":"b3dac88c5563a3b4a70345b1face75121ebc12cd6a2b839638a7a313a73d898c"}
Sep 10 21:16:49.499736 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:49.499710    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv"
Sep 10 21:16:49.556928 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:49.556893    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccd5debd-9a05-4b64-bf4e-088cd1a600d0-maas-api-service-ca\") pod \"ccd5debd-9a05-4b64-bf4e-088cd1a600d0\" (UID: \"ccd5debd-9a05-4b64-bf4e-088cd1a600d0\") "
Sep 10 21:16:49.557278 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:49.557255    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccd5debd-9a05-4b64-bf4e-088cd1a600d0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ccd5debd-9a05-4b64-bf4e-088cd1a600d0" (UID: "ccd5debd-9a05-4b64-bf4e-088cd1a600d0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:16:49.658016 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:49.657943    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccd5debd-9a05-4b64-bf4e-088cd1a600d0-maas-api-service-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:16:49.784693 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:49.784660    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:49.884471 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:49.884440    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:50.373763 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.373732    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv"
Sep 10 21:16:50.373938 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.373763    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv" event={"ID":"ccd5debd-9a05-4b64-bf4e-088cd1a600d0","Type":"ContainerDied","Data":"b3dac88c5563a3b4a70345b1face75121ebc12cd6a2b839638a7a313a73d898c"}
Sep 10 21:16:50.373938 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.373807    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3dac88c5563a3b4a70345b1face75121ebc12cd6a2b839638a7a313a73d898c"
Sep 10 21:16:50.916762 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.916485    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"]
Sep 10 21:16:50.917244 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.917092    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccd5debd-9a05-4b64-bf4e-088cd1a600d0" containerName="revoke-keys"
Sep 10 21:16:50.917244 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.917114    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccd5debd-9a05-4b64-bf4e-088cd1a600d0" containerName="revoke-keys"
Sep 10 21:16:50.917244 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.917220    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccd5debd-9a05-4b64-bf4e-088cd1a600d0" containerName="revoke-keys"
Sep 10 21:16:50.920376 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.920351    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:50.924522 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.924499    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-0ac302-serving-cert\""
Sep 10 21:16:50.924648 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.924531    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:16:50.924648 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.924505    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-njbnl\""
Sep 10 21:16:50.924648 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.924608    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:16:50.924828 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.924501    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:16:50.932045 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.932020    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"]
Sep 10 21:16:50.947031 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.947002    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:50.971416 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.971385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:50.971570 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.971470    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:50.971570 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.971524    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdhjq\" (UniqueName: \"kubernetes.io/projected/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-kube-api-access-xdhjq\") pod \"maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:50.971655 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:50.971575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:51.047330 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.047298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:51.072655 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.072617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:51.072802 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.072699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:51.072802 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.072755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdhjq\" (UniqueName: \"kubernetes.io/projected/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-kube-api-access-xdhjq\") pod \"maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:51.072802 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:16:51.072770    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-serving-cert: secret "maas-api-e2e-shared-a-w3-0ac302-serving-cert" not found
Sep 10 21:16:51.072977 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.072814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:51.072977 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:16:51.072854    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-maas-api-tls podName:2963a0f9-9413-4d70-b5d1-d4d79b8c37e5 nodeName:}" failed. No retries permitted until 2026-09-10 21:16:51.572830861 +0000 UTC m=+1105.252329694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-maas-api-tls") pod "maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff" (UID: "2963a0f9-9413-4d70-b5d1-d4d79b8c37e5") : secret "maas-api-e2e-shared-a-w3-0ac302-serving-cert" not found
Sep 10 21:16:51.073493 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.073470    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:51.075253 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.075235    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:51.084099 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.084077    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdhjq\" (UniqueName: \"kubernetes.io/projected/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-kube-api-access-xdhjq\") pod \"maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:51.432418 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.432385    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:51.567286 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.567252    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:51.577004 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.576982    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:51.579599 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.579572    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:51.666741 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.666704    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:51.683817 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.683740    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"]
Sep 10 21:16:51.688482 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.688461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"
Sep 10 21:16:51.691199 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.691179    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-0ac302-dockercfg-mg858\""
Sep 10 21:16:51.691332 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.691183    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-0ac302\""
Sep 10 21:16:51.699377 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.699355    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"]
Sep 10 21:16:51.779053 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.779012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a14f6d0-eb22-4209-b663-3abf1479a686-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp\" (UID: \"8a14f6d0-eb22-4209-b663-3abf1479a686\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"
Sep 10 21:16:51.779226 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.779132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5xqb\" (UniqueName: \"kubernetes.io/projected/8a14f6d0-eb22-4209-b663-3abf1479a686-kube-api-access-b5xqb\") pod \"payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp\" (UID: \"8a14f6d0-eb22-4209-b663-3abf1479a686\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"
Sep 10 21:16:51.825837 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.825799    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"]
Sep 10 21:16:51.829182 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.829157    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"
Sep 10 21:16:51.834513 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.834494    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:51.841319 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.841292    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"]
Sep 10 21:16:51.880391 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.880357    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pmr4\" (UniqueName: \"kubernetes.io/projected/8db3a326-fe52-47a1-8378-3bd8aebd7ae8-kube-api-access-5pmr4\") pod \"payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9\" (UID: \"8db3a326-fe52-47a1-8378-3bd8aebd7ae8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"
Sep 10 21:16:51.880556 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.880422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a14f6d0-eb22-4209-b663-3abf1479a686-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp\" (UID: \"8a14f6d0-eb22-4209-b663-3abf1479a686\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"
Sep 10 21:16:51.880556 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.880463    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db3a326-fe52-47a1-8378-3bd8aebd7ae8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9\" (UID: \"8db3a326-fe52-47a1-8378-3bd8aebd7ae8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"
Sep 10 21:16:51.880634 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.880596    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5xqb\" (UniqueName: \"kubernetes.io/projected/8a14f6d0-eb22-4209-b663-3abf1479a686-kube-api-access-b5xqb\") pod \"payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp\" (UID: \"8a14f6d0-eb22-4209-b663-3abf1479a686\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"
Sep 10 21:16:51.881144 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.881126    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a14f6d0-eb22-4209-b663-3abf1479a686-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp\" (UID: \"8a14f6d0-eb22-4209-b663-3abf1479a686\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"
Sep 10 21:16:51.890195 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.890173    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5xqb\" (UniqueName: \"kubernetes.io/projected/8a14f6d0-eb22-4209-b663-3abf1479a686-kube-api-access-b5xqb\") pod \"payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp\" (UID: \"8a14f6d0-eb22-4209-b663-3abf1479a686\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"
Sep 10 21:16:51.981343 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.981246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pmr4\" (UniqueName: \"kubernetes.io/projected/8db3a326-fe52-47a1-8378-3bd8aebd7ae8-kube-api-access-5pmr4\") pod \"payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9\" (UID: \"8db3a326-fe52-47a1-8378-3bd8aebd7ae8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"
Sep 10 21:16:51.981343 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.981305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db3a326-fe52-47a1-8378-3bd8aebd7ae8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9\" (UID: \"8db3a326-fe52-47a1-8378-3bd8aebd7ae8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"
Sep 10 21:16:51.981943 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.981923    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db3a326-fe52-47a1-8378-3bd8aebd7ae8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9\" (UID: \"8db3a326-fe52-47a1-8378-3bd8aebd7ae8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"
Sep 10 21:16:51.990486 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.990464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pmr4\" (UniqueName: \"kubernetes.io/projected/8db3a326-fe52-47a1-8378-3bd8aebd7ae8-kube-api-access-5pmr4\") pod \"payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9\" (UID: \"8db3a326-fe52-47a1-8378-3bd8aebd7ae8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"
Sep 10 21:16:51.999250 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:51.999227    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"
Sep 10 21:16:52.113729 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:52.113702    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"]
Sep 10 21:16:52.115514 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:16:52.115477    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2963a0f9_9413_4d70_b5d1_d4d79b8c37e5.slice/crio-09a693ae0418e86b35ab06be5455c3ea7785f106f1c84a26cb3f297e11e56925 WatchSource:0}: Error finding container 09a693ae0418e86b35ab06be5455c3ea7785f106f1c84a26cb3f297e11e56925: Status 404 returned error can't find the container with id 09a693ae0418e86b35ab06be5455c3ea7785f106f1c84a26cb3f297e11e56925
Sep 10 21:16:52.139182 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:52.139160    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"
Sep 10 21:16:52.276517 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:52.276351    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"]
Sep 10 21:16:52.279133 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:16:52.279103    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a14f6d0_eb22_4209_b663_3abf1479a686.slice/crio-a6558347a50129b3e99f7b4f739358ed3a09f587fb0a208680bff55a9fafbcab WatchSource:0}: Error finding container a6558347a50129b3e99f7b4f739358ed3a09f587fb0a208680bff55a9fafbcab: Status 404 returned error can't find the container with id a6558347a50129b3e99f7b4f739358ed3a09f587fb0a208680bff55a9fafbcab
Sep 10 21:16:52.386124 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:52.386086    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp" event={"ID":"8a14f6d0-eb22-4209-b663-3abf1479a686","Type":"ContainerStarted","Data":"0678cc9ba80e23c9c0605a2710ae37c34eef79e9c247c5c2affcfc6dd7402659"}
Sep 10 21:16:52.386290 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:52.386144    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp" event={"ID":"8a14f6d0-eb22-4209-b663-3abf1479a686","Type":"ContainerStarted","Data":"a6558347a50129b3e99f7b4f739358ed3a09f587fb0a208680bff55a9fafbcab"}
Sep 10 21:16:52.386290 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:52.386231    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"
Sep 10 21:16:52.387408 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:52.387381    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff" event={"ID":"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5","Type":"ContainerStarted","Data":"09a693ae0418e86b35ab06be5455c3ea7785f106f1c84a26cb3f297e11e56925"}
Sep 10 21:16:52.412079 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:52.412009    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp" podStartSLOduration=1.411988961 podStartE2EDuration="1.411988961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:52.409102009 +0000 UTC m=+1106.088600859" watchObservedRunningTime="2026-09-10 21:16:52.411988961 +0000 UTC m=+1106.091487811"
Sep 10 21:16:52.415839 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:52.415816    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"]
Sep 10 21:16:52.418015 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:16:52.417992    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db3a326_fe52_47a1_8378_3bd8aebd7ae8.slice/crio-85ed56c79cd99580df47a102d7b3e95feb28073bb8a7813a5e6b9f8c9041fdf7 WatchSource:0}: Error finding container 85ed56c79cd99580df47a102d7b3e95feb28073bb8a7813a5e6b9f8c9041fdf7: Status 404 returned error can't find the container with id 85ed56c79cd99580df47a102d7b3e95feb28073bb8a7813a5e6b9f8c9041fdf7
Sep 10 21:16:53.397770 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:53.397733    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9" event={"ID":"8db3a326-fe52-47a1-8378-3bd8aebd7ae8","Type":"ContainerStarted","Data":"a4f8b99c06ec0f347172e162de0129278dab20e48db2bb351d5523582ab958e1"}
Sep 10 21:16:53.398301 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:53.397801    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9" event={"ID":"8db3a326-fe52-47a1-8378-3bd8aebd7ae8","Type":"ContainerStarted","Data":"85ed56c79cd99580df47a102d7b3e95feb28073bb8a7813a5e6b9f8c9041fdf7"}
Sep 10 21:16:53.398301 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:53.398048    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"
Sep 10 21:16:53.417616 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:53.417555    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9" podStartSLOduration=2.417533566 podStartE2EDuration="2.417533566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:53.414387341 +0000 UTC m=+1107.093886194" watchObservedRunningTime="2026-09-10 21:16:53.417533566 +0000 UTC m=+1107.097032419"
Sep 10 21:16:54.455141 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:54.455106    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:16:55.411956 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:55.411916    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff" event={"ID":"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5","Type":"ContainerStarted","Data":"6d4f380cd835a5493e2854a9df550f82177fb84856c914802d1dd01ec7414a65"}
Sep 10 21:16:55.412155 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:55.412074    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:16:55.431821 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:55.431755    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff" podStartSLOduration=3.048529125 podStartE2EDuration="5.431735475s" totalImagesPullingTime="2.38320635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:55.430776184 +0000 UTC m=+1109.110275037" watchObservedRunningTime="2026-09-10 21:16:55.431735475 +0000 UTC m=+1109.111234326"
Sep 10 21:16:57.061174 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:57.061138    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:57.638679 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:57.638643    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:57.677497 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:57.677463    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"]
Sep 10 21:16:57.677734 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:57.677704    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c" podUID="a9d28fcf-f470-4092-9e7c-45bcfb7ea95c" containerName="payload-pre-processing" containerID="cri-o://d78358612132093334b91200893328dfbe61cbcf4f2a2b999128a37a2e98f43e" gracePeriod=30
Sep 10 21:16:57.738601 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:57.738567    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:57.834445 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:57.834414    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv"]
Sep 10 21:16:57.842440 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:57.842413    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:57.843226 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:57.843199    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-170a718d-6gplv"]
Sep 10 21:16:57.903372 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:57.903284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:57.933660 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:57.933635    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"
Sep 10 21:16:58.038763 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.038728    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9d28fcf-f470-4092-9e7c-45bcfb7ea95c-plugins-config-volume\") pod \"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c\" (UID: \"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c\") "
Sep 10 21:16:58.038763 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.038769    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d95p5\" (UniqueName: \"kubernetes.io/projected/a9d28fcf-f470-4092-9e7c-45bcfb7ea95c-kube-api-access-d95p5\") pod \"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c\" (UID: \"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c\") "
Sep 10 21:16:58.039117 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.039087    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9d28fcf-f470-4092-9e7c-45bcfb7ea95c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a9d28fcf-f470-4092-9e7c-45bcfb7ea95c" (UID: "a9d28fcf-f470-4092-9e7c-45bcfb7ea95c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:16:58.040889 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.040868    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9d28fcf-f470-4092-9e7c-45bcfb7ea95c-kube-api-access-d95p5" (OuterVolumeSpecName: "kube-api-access-d95p5") pod "a9d28fcf-f470-4092-9e7c-45bcfb7ea95c" (UID: "a9d28fcf-f470-4092-9e7c-45bcfb7ea95c"). InnerVolumeSpecName "kube-api-access-d95p5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:16:58.139623 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.139586    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9d28fcf-f470-4092-9e7c-45bcfb7ea95c-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:16:58.139623 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.139617    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d95p5\" (UniqueName: \"kubernetes.io/projected/a9d28fcf-f470-4092-9e7c-45bcfb7ea95c-kube-api-access-d95p5\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:16:58.237925 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.237838    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:16:58.428708 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.428672    2580 generic.go:356] "Generic (PLEG): container finished" podID="a9d28fcf-f470-4092-9e7c-45bcfb7ea95c" containerID="d78358612132093334b91200893328dfbe61cbcf4f2a2b999128a37a2e98f43e" exitCode=0
Sep 10 21:16:58.428918 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.428719    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"
Sep 10 21:16:58.428918 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.428765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c" event={"ID":"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c","Type":"ContainerDied","Data":"d78358612132093334b91200893328dfbe61cbcf4f2a2b999128a37a2e98f43e"}
Sep 10 21:16:58.428918 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.428830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c" event={"ID":"a9d28fcf-f470-4092-9e7c-45bcfb7ea95c","Type":"ContainerDied","Data":"a22707aa012dde3ec4c7793a78e8fc18af268b9056aed5dae3ea8d6b8d02be82"}
Sep 10 21:16:58.428918 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.428862    2580 scope.go:117] "RemoveContainer" containerID="d78358612132093334b91200893328dfbe61cbcf4f2a2b999128a37a2e98f43e"
Sep 10 21:16:58.439383 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.439362    2580 scope.go:117] "RemoveContainer" containerID="d78358612132093334b91200893328dfbe61cbcf4f2a2b999128a37a2e98f43e"
Sep 10 21:16:58.439660 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:16:58.439641    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d78358612132093334b91200893328dfbe61cbcf4f2a2b999128a37a2e98f43e\": container with ID starting with d78358612132093334b91200893328dfbe61cbcf4f2a2b999128a37a2e98f43e not found: ID does not exist" containerID="d78358612132093334b91200893328dfbe61cbcf4f2a2b999128a37a2e98f43e"
Sep 10 21:16:58.439717 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.439672    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d78358612132093334b91200893328dfbe61cbcf4f2a2b999128a37a2e98f43e"} err="failed to get container status \"d78358612132093334b91200893328dfbe61cbcf4f2a2b999128a37a2e98f43e\": rpc error: code = NotFound desc = could not find container \"d78358612132093334b91200893328dfbe61cbcf4f2a2b999128a37a2e98f43e\": container with ID starting with d78358612132093334b91200893328dfbe61cbcf4f2a2b999128a37a2e98f43e not found: ID does not exist"
Sep 10 21:16:58.452115 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.452091    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"]
Sep 10 21:16:58.458920 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.458899    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-170a718d-6c7f874cc4-d4n4c"]
Sep 10 21:16:58.957685 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.957653    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9d28fcf-f470-4092-9e7c-45bcfb7ea95c" path="/var/lib/kubelet/pods/a9d28fcf-f470-4092-9e7c-45bcfb7ea95c/volumes"
Sep 10 21:16:58.957998 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:16:58.957986    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccd5debd-9a05-4b64-bf4e-088cd1a600d0" path="/var/lib/kubelet/pods/ccd5debd-9a05-4b64-bf4e-088cd1a600d0/volumes"
Sep 10 21:17:01.423376 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:01.423342    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:17:02.230565 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:02.230532    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:17:03.400562 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:03.400528    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"
Sep 10 21:17:04.407245 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:04.407216    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"
Sep 10 21:17:04.562046 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:04.562012    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:04.624641 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:04.624594    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:04.869625 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:04.869591    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:05.216307 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:05.216201    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:05.316804 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:05.316761    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:05.452351 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:05.452279    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:11.499508 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:11.499467    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:11.598792 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:11.598753    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:11.706342 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:11.706284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:11.891104 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:11.891037    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:12.076950 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:12.076911    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:14.807840 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:14.807798    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:15.148545 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:15.148462    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:15.431245 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:15.431161    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:15.783737 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:15.783645    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:18.228049 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:18.228010    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:18.328794 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:18.328765    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:18.428556 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:18.428522    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:18.494247 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:18.494179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:19.204962 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:19.204928    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:19.558902 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:19.558871    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:19.623664 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:19.623627    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:19.917591 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:19.917515    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:20.131529 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.131502    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:20.220608 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.220533    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:20.331304 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.331277    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"]
Sep 10 21:17:20.331762 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.331745    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9d28fcf-f470-4092-9e7c-45bcfb7ea95c" containerName="payload-pre-processing"
Sep 10 21:17:20.331825 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.331764    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9d28fcf-f470-4092-9e7c-45bcfb7ea95c" containerName="payload-pre-processing"
Sep 10 21:17:20.331870 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.331855    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9d28fcf-f470-4092-9e7c-45bcfb7ea95c" containerName="payload-pre-processing"
Sep 10 21:17:20.336401 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.336383    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"
Sep 10 21:17:20.339557 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.339529    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-16bbec\""
Sep 10 21:17:20.339685 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.339572    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-16bbec-dockercfg-ttcd7\""
Sep 10 21:17:20.347263 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.347241    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"]
Sep 10 21:17:20.394864 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.394839    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:20.444821 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.444794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hrml\" (UniqueName: \"kubernetes.io/projected/1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835-kube-api-access-9hrml\") pod \"payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt\" (UID: \"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"
Sep 10 21:17:20.444941 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.444886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt\" (UID: \"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"
Sep 10 21:17:20.545370 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.545343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hrml\" (UniqueName: \"kubernetes.io/projected/1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835-kube-api-access-9hrml\") pod \"payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt\" (UID: \"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"
Sep 10 21:17:20.545516 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.545401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt\" (UID: \"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"
Sep 10 21:17:20.546034 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.546014    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt\" (UID: \"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"
Sep 10 21:17:20.554126 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.554102    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hrml\" (UniqueName: \"kubernetes.io/projected/1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835-kube-api-access-9hrml\") pod \"payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt\" (UID: \"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"
Sep 10 21:17:20.648038 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.648006    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"
Sep 10 21:17:20.920580 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:20.920554    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"]
Sep 10 21:17:20.922522 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:17:20.922495    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f04b9d4_3ff6_4d1f_ac4d_fe52a2175835.slice/crio-7e8a20ede7c35774f114b111910c30e51b859bb20d7c41241fb29c1c2dc979dc WatchSource:0}: Error finding container 7e8a20ede7c35774f114b111910c30e51b859bb20d7c41241fb29c1c2dc979dc: Status 404 returned error can't find the container with id 7e8a20ede7c35774f114b111910c30e51b859bb20d7c41241fb29c1c2dc979dc
Sep 10 21:17:21.556488 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:21.556453    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt" event={"ID":"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835","Type":"ContainerStarted","Data":"08c2a62851b65a350fd790d0010340ae288a2b260f80c330758371ee25088146"}
Sep 10 21:17:21.556651 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:21.556514    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt" event={"ID":"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835","Type":"ContainerStarted","Data":"7e8a20ede7c35774f114b111910c30e51b859bb20d7c41241fb29c1c2dc979dc"}
Sep 10 21:17:21.556651 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:21.556571    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"
Sep 10 21:17:21.575162 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:21.575114    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt" podStartSLOduration=1.575099749 podStartE2EDuration="1.575099749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:21.57415628 +0000 UTC m=+1135.253655133" watchObservedRunningTime="2026-09-10 21:17:21.575099749 +0000 UTC m=+1135.254598599"
Sep 10 21:17:22.452430 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:22.452380    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:22.521099 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:22.521051    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:23.744562 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.744520    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"]
Sep 10 21:17:23.749137 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.749116    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:23.751913 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.751881    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-501c57-serving-cert\""
Sep 10 21:17:23.759662 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.759643    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"]
Sep 10 21:17:23.774859 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.774831    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:23.872196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.872160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:23.876469 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.876443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:23.876574 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.876485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:23.876574 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.876522    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fptgn\" (UniqueName: \"kubernetes.io/projected/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-kube-api-access-fptgn\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:23.876668 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.876649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:23.977504 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.977478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:23.977636 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.977521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:23.977636 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.977541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:23.977636 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.977565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptgn\" (UniqueName: \"kubernetes.io/projected/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-kube-api-access-fptgn\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:23.977778 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:17:23.977628    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-serving-cert: secret "maas-api-e2e-shared-a-w2-501c57-serving-cert" not found
Sep 10 21:17:23.977778 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:17:23.977698    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-maas-api-tls podName:67da1ae0-b6a3-4e04-b5b4-23c0bbf830db nodeName:}" failed. No retries permitted until 2026-09-10 21:17:24.477678771 +0000 UTC m=+1138.157177601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-maas-api-tls") pod "maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk" (UID: "67da1ae0-b6a3-4e04-b5b4-23c0bbf830db") : secret "maas-api-e2e-shared-a-w2-501c57-serving-cert" not found
Sep 10 21:17:23.978258 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.978241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:23.980038 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.980007    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:23.986156 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:23.986136    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fptgn\" (UniqueName: \"kubernetes.io/projected/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-kube-api-access-fptgn\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:24.233982 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.233950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:24.332288 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.332248    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:24.374625 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.374598    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"]
Sep 10 21:17:24.378443 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.378422    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"
Sep 10 21:17:24.381184 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.381157    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-501c57\""
Sep 10 21:17:24.381306 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.381181    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-501c57-dockercfg-xf854\""
Sep 10 21:17:24.389196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.389173    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"]
Sep 10 21:17:24.449484 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.449452    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:17:24.482231 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.482203    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96kx4\" (UniqueName: \"kubernetes.io/projected/cdf92600-aa61-4503-ba65-01a2d30e3ff3-kube-api-access-96kx4\") pod \"payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb\" (UID: \"cdf92600-aa61-4503-ba65-01a2d30e3ff3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"
Sep 10 21:17:24.482375 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.482241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdf92600-aa61-4503-ba65-01a2d30e3ff3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb\" (UID: \"cdf92600-aa61-4503-ba65-01a2d30e3ff3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"
Sep 10 21:17:24.482375 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.482273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:24.482375 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:17:24.482356    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-serving-cert: secret "maas-api-e2e-shared-a-w2-501c57-serving-cert" not found
Sep 10 21:17:24.482546 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:17:24.482414    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-maas-api-tls podName:67da1ae0-b6a3-4e04-b5b4-23c0bbf830db nodeName:}" failed. No retries permitted until 2026-09-10 21:17:25.482398288 +0000 UTC m=+1139.161897116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-maas-api-tls") pod "maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk" (UID: "67da1ae0-b6a3-4e04-b5b4-23c0bbf830db") : secret "maas-api-e2e-shared-a-w2-501c57-serving-cert" not found
Sep 10 21:17:24.583638 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.583610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96kx4\" (UniqueName: \"kubernetes.io/projected/cdf92600-aa61-4503-ba65-01a2d30e3ff3-kube-api-access-96kx4\") pod \"payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb\" (UID: \"cdf92600-aa61-4503-ba65-01a2d30e3ff3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"
Sep 10 21:17:24.583781 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.583649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdf92600-aa61-4503-ba65-01a2d30e3ff3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb\" (UID: \"cdf92600-aa61-4503-ba65-01a2d30e3ff3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"
Sep 10 21:17:24.584520 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.584494    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdf92600-aa61-4503-ba65-01a2d30e3ff3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb\" (UID: \"cdf92600-aa61-4503-ba65-01a2d30e3ff3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"
Sep 10 21:17:24.590702 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.590682    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:24.596679 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.596658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96kx4\" (UniqueName: \"kubernetes.io/projected/cdf92600-aa61-4503-ba65-01a2d30e3ff3-kube-api-access-96kx4\") pod \"payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb\" (UID: \"cdf92600-aa61-4503-ba65-01a2d30e3ff3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"
Sep 10 21:17:24.690579 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.690548    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"
Sep 10 21:17:24.959956 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:24.959933    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"]
Sep 10 21:17:24.961933 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:17:24.961911    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf92600_aa61_4503_ba65_01a2d30e3ff3.slice/crio-aeb94ac72a4a0a0d9d292e26ab57780fbdc517a56a0635627b3bae523bcdaa77 WatchSource:0}: Error finding container aeb94ac72a4a0a0d9d292e26ab57780fbdc517a56a0635627b3bae523bcdaa77: Status 404 returned error can't find the container with id aeb94ac72a4a0a0d9d292e26ab57780fbdc517a56a0635627b3bae523bcdaa77
Sep 10 21:17:25.491971 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:25.491929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:25.494442 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:25.494409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:25.562321 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:25.562283    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:25.578951 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:25.578925    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb" event={"ID":"cdf92600-aa61-4503-ba65-01a2d30e3ff3","Type":"ContainerStarted","Data":"2e161306fa20fbc60c4fb218e93ed1bc5a56c536b656fff80c97e1dc777fc5cb"}
Sep 10 21:17:25.579092 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:25.578975    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb" event={"ID":"cdf92600-aa61-4503-ba65-01a2d30e3ff3","Type":"ContainerStarted","Data":"aeb94ac72a4a0a0d9d292e26ab57780fbdc517a56a0635627b3bae523bcdaa77"}
Sep 10 21:17:25.579092 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:25.579044    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"
Sep 10 21:17:25.598211 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:25.598165    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb" podStartSLOduration=1.598147429 podStartE2EDuration="1.598147429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:25.596694806 +0000 UTC m=+1139.276193669" watchObservedRunningTime="2026-09-10 21:17:25.598147429 +0000 UTC m=+1139.277646279"
Sep 10 21:17:25.835172 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:25.835145    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"]
Sep 10 21:17:25.837143 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:17:25.837116    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67da1ae0_b6a3_4e04_b5b4_23c0bbf830db.slice/crio-102875abce80b55acc8b5b730172096ac509ed70cc9799dd86c32ca8ed8613d4 WatchSource:0}: Error finding container 102875abce80b55acc8b5b730172096ac509ed70cc9799dd86c32ca8ed8613d4: Status 404 returned error can't find the container with id 102875abce80b55acc8b5b730172096ac509ed70cc9799dd86c32ca8ed8613d4
Sep 10 21:17:26.163237 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:26.163211    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:26.264712 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:26.264681    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:26.586040 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:26.586008    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk" event={"ID":"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db","Type":"ContainerStarted","Data":"04c15f0fdd9c88dc031cc12810f6147e23e9768dc2bc2d2a9c8534ef8c2f0c83"}
Sep 10 21:17:26.586211 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:26.586078    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk" event={"ID":"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db","Type":"ContainerStarted","Data":"102875abce80b55acc8b5b730172096ac509ed70cc9799dd86c32ca8ed8613d4"}
Sep 10 21:17:26.586211 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:26.586109    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:26.613334 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:26.613280    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk" podStartSLOduration=3.309544588 podStartE2EDuration="3.613263546s" totalImagesPullingTime="303.718958ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:26.604577602 +0000 UTC m=+1140.284076455" watchObservedRunningTime="2026-09-10 21:17:26.613263546 +0000 UTC m=+1140.292762396"
Sep 10 21:17:32.563824 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:32.563789    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"
Sep 10 21:17:32.598123 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:32.598099    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:17:36.588470 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:36.588435    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"
Sep 10 21:17:37.149337 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:37.149304    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:17:40.599665 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:40.599631    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:43.424776 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:43.424737    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:46.000924 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:46.000889    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:47.457377 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:47.457336    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:47.802696 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:47.802662    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:48.145761 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:48.145663    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:48.452703 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:48.452614    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:50.089711 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:50.089632    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:51.242760 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:51.242712    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:51.329141 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:51.329105    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:52.396880 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:52.396847    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:52.497508 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:52.497470    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:52.811604 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:52.811570    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:52.912068 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:52.912038    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:52.981023 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:52.980986    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:17:52.982564 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:52.982542    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"]
Sep 10 21:17:52.989887 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:52.989872    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"
Sep 10 21:17:52.993174 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:52.993148    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-4ad710\""
Sep 10 21:17:52.993316 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:52.993214    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-4ad710-dockercfg-fz896\""
Sep 10 21:17:52.999604 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:52.999583    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"]
Sep 10 21:17:53.134090 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:53.133994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv6zn\" (UniqueName: \"kubernetes.io/projected/aa04180c-ace2-4253-8200-133d7db17b21-kube-api-access-xv6zn\") pod \"payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx\" (UID: \"aa04180c-ace2-4253-8200-133d7db17b21\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"
Sep 10 21:17:53.134255 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:53.134103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa04180c-ace2-4253-8200-133d7db17b21-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx\" (UID: \"aa04180c-ace2-4253-8200-133d7db17b21\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"
Sep 10 21:17:53.234858 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:53.234828    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa04180c-ace2-4253-8200-133d7db17b21-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx\" (UID: \"aa04180c-ace2-4253-8200-133d7db17b21\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"
Sep 10 21:17:53.235035 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:53.234897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv6zn\" (UniqueName: \"kubernetes.io/projected/aa04180c-ace2-4253-8200-133d7db17b21-kube-api-access-xv6zn\") pod \"payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx\" (UID: \"aa04180c-ace2-4253-8200-133d7db17b21\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"
Sep 10 21:17:53.235627 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:53.235608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa04180c-ace2-4253-8200-133d7db17b21-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx\" (UID: \"aa04180c-ace2-4253-8200-133d7db17b21\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"
Sep 10 21:17:53.244293 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:53.244274    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv6zn\" (UniqueName: \"kubernetes.io/projected/aa04180c-ace2-4253-8200-133d7db17b21-kube-api-access-xv6zn\") pod \"payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx\" (UID: \"aa04180c-ace2-4253-8200-133d7db17b21\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"
Sep 10 21:17:53.301279 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:53.301249    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"
Sep 10 21:17:53.443036 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:53.442959    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:17:53.573597 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:53.573569    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"]
Sep 10 21:17:53.575553 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:17:53.575528    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa04180c_ace2_4253_8200_133d7db17b21.slice/crio-3b83cfadd3741fc8cd765d5eb30c626c1add83393dc72ff56c6ce185679ff856 WatchSource:0}: Error finding container 3b83cfadd3741fc8cd765d5eb30c626c1add83393dc72ff56c6ce185679ff856: Status 404 returned error can't find the container with id 3b83cfadd3741fc8cd765d5eb30c626c1add83393dc72ff56c6ce185679ff856
Sep 10 21:17:53.734742 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:53.734656    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx" event={"ID":"aa04180c-ace2-4253-8200-133d7db17b21","Type":"ContainerStarted","Data":"31f6438a599b883ec57296311682e45c6ba4ae3e5958976982a1e26c8a63dbfb"}
Sep 10 21:17:53.734742 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:53.734713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx" event={"ID":"aa04180c-ace2-4253-8200-133d7db17b21","Type":"ContainerStarted","Data":"3b83cfadd3741fc8cd765d5eb30c626c1add83393dc72ff56c6ce185679ff856"}
Sep 10 21:17:53.734970 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:53.734746    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"
Sep 10 21:17:53.760069 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:17:53.760014    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx" podStartSLOduration=1.759998 podStartE2EDuration="1.759998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:53.757369245 +0000 UTC m=+1167.436868110" watchObservedRunningTime="2026-09-10 21:17:53.759998 +0000 UTC m=+1167.439496891"
Sep 10 21:18:04.742032 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:04.742003    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"
Sep 10 21:18:06.532518 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:06.532481    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:18:08.427038 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:08.427002    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:08.481426 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:08.481396    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:16.155977 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:16.155940    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:26.662875 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:26.662833    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:27.001839 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:27.001737    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:27.238616 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:27.238571    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:30.421832 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:30.421794    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:30.515350 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:30.515310    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:31.548648 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.548612    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"]
Sep 10 21:18:31.554602 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.554580    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:31.557308 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.557276    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7f376d02-serving-cert\""
Sep 10 21:18:31.563242 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.563211    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"]
Sep 10 21:18:31.581317 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.581273    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:31.680844 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.680810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:31.688863 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.688832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd7c48cc-79b3-4301-90d2-71e948271f65-openshift-service-ca\") pod \"maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:31.689002 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.688886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd7c48cc-79b3-4301-90d2-71e948271f65-maas-api-tls\") pod \"maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:31.689002 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.688965    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd7c48cc-79b3-4301-90d2-71e948271f65-metrics-certs\") pod \"maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:31.689146 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.689005    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnkz4\" (UniqueName: \"kubernetes.io/projected/fd7c48cc-79b3-4301-90d2-71e948271f65-kube-api-access-xnkz4\") pod \"maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:31.789719 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.789676    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd7c48cc-79b3-4301-90d2-71e948271f65-openshift-service-ca\") pod \"maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:31.789900 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.789732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd7c48cc-79b3-4301-90d2-71e948271f65-maas-api-tls\") pod \"maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:31.789900 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.789767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd7c48cc-79b3-4301-90d2-71e948271f65-metrics-certs\") pod \"maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:31.789900 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.789796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnkz4\" (UniqueName: \"kubernetes.io/projected/fd7c48cc-79b3-4301-90d2-71e948271f65-kube-api-access-xnkz4\") pod \"maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:31.790501 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.790473    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd7c48cc-79b3-4301-90d2-71e948271f65-openshift-service-ca\") pod \"maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:31.792333 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.792309    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd7c48cc-79b3-4301-90d2-71e948271f65-maas-api-tls\") pod \"maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:31.792430 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.792420    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd7c48cc-79b3-4301-90d2-71e948271f65-metrics-certs\") pod \"maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:31.798129 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.798103    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnkz4\" (UniqueName: \"kubernetes.io/projected/fd7c48cc-79b3-4301-90d2-71e948271f65-kube-api-access-xnkz4\") pod \"maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:31.804887 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.804816    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:31.868808 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:31.868773    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:32.155756 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.155728    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"]
Sep 10 21:18:32.157698 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:18:32.157672    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd7c48cc_79b3_4301_90d2_71e948271f65.slice/crio-2076fbe57c62e416cae7ecaadf1486645019fc72636a76b164086a0fb4d8b318 WatchSource:0}: Error finding container 2076fbe57c62e416cae7ecaadf1486645019fc72636a76b164086a0fb4d8b318: Status 404 returned error can't find the container with id 2076fbe57c62e416cae7ecaadf1486645019fc72636a76b164086a0fb4d8b318
Sep 10 21:18:32.259691 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.259658    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:32.350324 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.350289    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:32.405002 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.404912    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"]
Sep 10 21:18:32.410300 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.410276    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"
Sep 10 21:18:32.414634 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.414175    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-7f376d02\""
Sep 10 21:18:32.414634 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.414403    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-7f376d02-dockercfg-2xq6j\""
Sep 10 21:18:32.422470 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.422444    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"]
Sep 10 21:18:32.498234 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.498193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfqbs\" (UniqueName: \"kubernetes.io/projected/a1e1186d-fb7c-483a-bf39-88d90c57e821-kube-api-access-gfqbs\") pod \"payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf\" (UID: \"a1e1186d-fb7c-483a-bf39-88d90c57e821\") " pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"
Sep 10 21:18:32.498390 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.498316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1e1186d-fb7c-483a-bf39-88d90c57e821-plugins-config-volume\") pod \"payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf\" (UID: \"a1e1186d-fb7c-483a-bf39-88d90c57e821\") " pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"
Sep 10 21:18:32.599322 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.599284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1e1186d-fb7c-483a-bf39-88d90c57e821-plugins-config-volume\") pod \"payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf\" (UID: \"a1e1186d-fb7c-483a-bf39-88d90c57e821\") " pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"
Sep 10 21:18:32.599699 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.599400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfqbs\" (UniqueName: \"kubernetes.io/projected/a1e1186d-fb7c-483a-bf39-88d90c57e821-kube-api-access-gfqbs\") pod \"payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf\" (UID: \"a1e1186d-fb7c-483a-bf39-88d90c57e821\") " pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"
Sep 10 21:18:32.599950 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.599930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1e1186d-fb7c-483a-bf39-88d90c57e821-plugins-config-volume\") pod \"payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf\" (UID: \"a1e1186d-fb7c-483a-bf39-88d90c57e821\") " pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"
Sep 10 21:18:32.607642 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.607610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfqbs\" (UniqueName: \"kubernetes.io/projected/a1e1186d-fb7c-483a-bf39-88d90c57e821-kube-api-access-gfqbs\") pod \"payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf\" (UID: \"a1e1186d-fb7c-483a-bf39-88d90c57e821\") " pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"
Sep 10 21:18:32.726838 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.726751    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"
Sep 10 21:18:32.959349 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.959310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf" event={"ID":"fd7c48cc-79b3-4301-90d2-71e948271f65","Type":"ContainerStarted","Data":"9af8978930a9b72e17759d22a9d21b776b61cc139b534c3c0d6e940abe5e7cc6"}
Sep 10 21:18:32.959542 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.959372    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf" event={"ID":"fd7c48cc-79b3-4301-90d2-71e948271f65","Type":"ContainerStarted","Data":"2076fbe57c62e416cae7ecaadf1486645019fc72636a76b164086a0fb4d8b318"}
Sep 10 21:18:32.959542 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.959404    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:32.977392 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:32.977298    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf" podStartSLOduration=1.39892755 podStartE2EDuration="1.977282168s" totalImagesPullingTime="578.354618ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:18:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:18:32.976852047 +0000 UTC m=+1206.656350907" watchObservedRunningTime="2026-09-10 21:18:32.977282168 +0000 UTC m=+1206.656781034"
Sep 10 21:18:33.003741 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:33.003717    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"]
Sep 10 21:18:33.006311 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:18:33.006284    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1e1186d_fb7c_483a_bf39_88d90c57e821.slice/crio-19f7f15fb41cb74d11ac7fe62b5a832a17323564aef26448cf056f292c1d5c3d WatchSource:0}: Error finding container 19f7f15fb41cb74d11ac7fe62b5a832a17323564aef26448cf056f292c1d5c3d: Status 404 returned error can't find the container with id 19f7f15fb41cb74d11ac7fe62b5a832a17323564aef26448cf056f292c1d5c3d
Sep 10 21:18:33.516848 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:33.516805    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:33.966106 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:33.966035    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf" event={"ID":"a1e1186d-fb7c-483a-bf39-88d90c57e821","Type":"ContainerStarted","Data":"8af92a74f303bc7baf7b2a8348f9bcdd3ffbf86176478c51901a7aa3199a873e"}
Sep 10 21:18:33.966519 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:33.966249    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"
Sep 10 21:18:33.966519 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:33.966280    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf" event={"ID":"a1e1186d-fb7c-483a-bf39-88d90c57e821","Type":"ContainerStarted","Data":"19f7f15fb41cb74d11ac7fe62b5a832a17323564aef26448cf056f292c1d5c3d"}
Sep 10 21:18:33.985710 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:33.985662    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf" podStartSLOduration=1.985646306 podStartE2EDuration="1.985646306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:18:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:18:33.983844286 +0000 UTC m=+1207.663343168" watchObservedRunningTime="2026-09-10 21:18:33.985646306 +0000 UTC m=+1207.665145156"
Sep 10 21:18:38.972192 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:38.972162    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:18:40.454855 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:40.454820    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:18:40.494362 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:40.494329    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:18:44.974900 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:44.974867    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"
Sep 10 21:18:53.737965 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:53.737932    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:18:55.448815 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:18:55.448775    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:00.763682 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:00.763652    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:05.400528 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:05.400491    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:07.574277 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:07.572274    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:19:25.932740 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:25.932653    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:19:40.464317 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:40.464281    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:43.634515 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:43.634478    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:19:44.274545 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:44.274507    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:44.376113 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:44.376080    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:55.796376 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:55.796338    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"]
Sep 10 21:19:55.797224 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:55.797165    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf" podUID="fd7c48cc-79b3-4301-90d2-71e948271f65" containerName="maas-api" containerID="cri-o://9af8978930a9b72e17759d22a9d21b776b61cc139b534c3c0d6e940abe5e7cc6" gracePeriod=30
Sep 10 21:19:55.822641 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:55.822606    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:55.853714 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:55.853681    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"]
Sep 10 21:19:55.853983 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:55.853936    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf" podUID="a1e1186d-fb7c-483a-bf39-88d90c57e821" containerName="payload-processing" containerID="cri-o://8af92a74f303bc7baf7b2a8348f9bcdd3ffbf86176478c51901a7aa3199a873e" gracePeriod=30
Sep 10 21:19:55.922650 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:55.922620    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:56.040944 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.040909    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:56.117757 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.117717    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:56.118780 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.118758    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:19:56.148070 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.148034    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"
Sep 10 21:19:56.183486 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.183461    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd7c48cc-79b3-4301-90d2-71e948271f65-metrics-certs\") pod \"fd7c48cc-79b3-4301-90d2-71e948271f65\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") "
Sep 10 21:19:56.183677 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.183499    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd7c48cc-79b3-4301-90d2-71e948271f65-maas-api-tls\") pod \"fd7c48cc-79b3-4301-90d2-71e948271f65\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") "
Sep 10 21:19:56.183677 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.183524    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfqbs\" (UniqueName: \"kubernetes.io/projected/a1e1186d-fb7c-483a-bf39-88d90c57e821-kube-api-access-gfqbs\") pod \"a1e1186d-fb7c-483a-bf39-88d90c57e821\" (UID: \"a1e1186d-fb7c-483a-bf39-88d90c57e821\") "
Sep 10 21:19:56.183677 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.183559    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnkz4\" (UniqueName: \"kubernetes.io/projected/fd7c48cc-79b3-4301-90d2-71e948271f65-kube-api-access-xnkz4\") pod \"fd7c48cc-79b3-4301-90d2-71e948271f65\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") "
Sep 10 21:19:56.183677 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.183659    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd7c48cc-79b3-4301-90d2-71e948271f65-openshift-service-ca\") pod \"fd7c48cc-79b3-4301-90d2-71e948271f65\" (UID: \"fd7c48cc-79b3-4301-90d2-71e948271f65\") "
Sep 10 21:19:56.183677 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.183678    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1e1186d-fb7c-483a-bf39-88d90c57e821-plugins-config-volume\") pod \"a1e1186d-fb7c-483a-bf39-88d90c57e821\" (UID: \"a1e1186d-fb7c-483a-bf39-88d90c57e821\") "
Sep 10 21:19:56.184182 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.184072    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd7c48cc-79b3-4301-90d2-71e948271f65-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fd7c48cc-79b3-4301-90d2-71e948271f65" (UID: "fd7c48cc-79b3-4301-90d2-71e948271f65"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:19:56.184292 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.184260    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1e1186d-fb7c-483a-bf39-88d90c57e821-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a1e1186d-fb7c-483a-bf39-88d90c57e821" (UID: "a1e1186d-fb7c-483a-bf39-88d90c57e821"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:19:56.186213 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.186176    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd7c48cc-79b3-4301-90d2-71e948271f65-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fd7c48cc-79b3-4301-90d2-71e948271f65" (UID: "fd7c48cc-79b3-4301-90d2-71e948271f65"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:19:56.186316 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.186210    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1e1186d-fb7c-483a-bf39-88d90c57e821-kube-api-access-gfqbs" (OuterVolumeSpecName: "kube-api-access-gfqbs") pod "a1e1186d-fb7c-483a-bf39-88d90c57e821" (UID: "a1e1186d-fb7c-483a-bf39-88d90c57e821"). InnerVolumeSpecName "kube-api-access-gfqbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:19:56.186316 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.186260    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd7c48cc-79b3-4301-90d2-71e948271f65-kube-api-access-xnkz4" (OuterVolumeSpecName: "kube-api-access-xnkz4") pod "fd7c48cc-79b3-4301-90d2-71e948271f65" (UID: "fd7c48cc-79b3-4301-90d2-71e948271f65"). InnerVolumeSpecName "kube-api-access-xnkz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:19:56.186398 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.186339    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd7c48cc-79b3-4301-90d2-71e948271f65-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fd7c48cc-79b3-4301-90d2-71e948271f65" (UID: "fd7c48cc-79b3-4301-90d2-71e948271f65"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:19:56.284807 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.284768    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd7c48cc-79b3-4301-90d2-71e948271f65-openshift-service-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:19:56.284807 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.284800    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1e1186d-fb7c-483a-bf39-88d90c57e821-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:19:56.284807 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.284810    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd7c48cc-79b3-4301-90d2-71e948271f65-metrics-certs\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:19:56.285044 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.284821    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd7c48cc-79b3-4301-90d2-71e948271f65-maas-api-tls\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:19:56.285044 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.284829    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfqbs\" (UniqueName: \"kubernetes.io/projected/a1e1186d-fb7c-483a-bf39-88d90c57e821-kube-api-access-gfqbs\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:19:56.285044 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.284839    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnkz4\" (UniqueName: \"kubernetes.io/projected/fd7c48cc-79b3-4301-90d2-71e948271f65-kube-api-access-xnkz4\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:19:56.426311 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.426234    2580 generic.go:356] "Generic (PLEG): container finished" podID="a1e1186d-fb7c-483a-bf39-88d90c57e821" containerID="8af92a74f303bc7baf7b2a8348f9bcdd3ffbf86176478c51901a7aa3199a873e" exitCode=0
Sep 10 21:19:56.426311 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.426281    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"
Sep 10 21:19:56.426311 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.426300    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf" event={"ID":"a1e1186d-fb7c-483a-bf39-88d90c57e821","Type":"ContainerDied","Data":"8af92a74f303bc7baf7b2a8348f9bcdd3ffbf86176478c51901a7aa3199a873e"}
Sep 10 21:19:56.426552 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.426355    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf" event={"ID":"a1e1186d-fb7c-483a-bf39-88d90c57e821","Type":"ContainerDied","Data":"19f7f15fb41cb74d11ac7fe62b5a832a17323564aef26448cf056f292c1d5c3d"}
Sep 10 21:19:56.426552 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.426386    2580 scope.go:117] "RemoveContainer" containerID="8af92a74f303bc7baf7b2a8348f9bcdd3ffbf86176478c51901a7aa3199a873e"
Sep 10 21:19:56.427731 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.427711    2580 generic.go:356] "Generic (PLEG): container finished" podID="fd7c48cc-79b3-4301-90d2-71e948271f65" containerID="9af8978930a9b72e17759d22a9d21b776b61cc139b534c3c0d6e940abe5e7cc6" exitCode=0
Sep 10 21:19:56.427843 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.427747    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"
Sep 10 21:19:56.427843 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.427756    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf" event={"ID":"fd7c48cc-79b3-4301-90d2-71e948271f65","Type":"ContainerDied","Data":"9af8978930a9b72e17759d22a9d21b776b61cc139b534c3c0d6e940abe5e7cc6"}
Sep 10 21:19:56.427843 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.427811    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf" event={"ID":"fd7c48cc-79b3-4301-90d2-71e948271f65","Type":"ContainerDied","Data":"2076fbe57c62e416cae7ecaadf1486645019fc72636a76b164086a0fb4d8b318"}
Sep 10 21:19:56.437210 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.437191    2580 scope.go:117] "RemoveContainer" containerID="8af92a74f303bc7baf7b2a8348f9bcdd3ffbf86176478c51901a7aa3199a873e"
Sep 10 21:19:56.437541 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:19:56.437524    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8af92a74f303bc7baf7b2a8348f9bcdd3ffbf86176478c51901a7aa3199a873e\": container with ID starting with 8af92a74f303bc7baf7b2a8348f9bcdd3ffbf86176478c51901a7aa3199a873e not found: ID does not exist" containerID="8af92a74f303bc7baf7b2a8348f9bcdd3ffbf86176478c51901a7aa3199a873e"
Sep 10 21:19:56.437590 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.437554    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8af92a74f303bc7baf7b2a8348f9bcdd3ffbf86176478c51901a7aa3199a873e"} err="failed to get container status \"8af92a74f303bc7baf7b2a8348f9bcdd3ffbf86176478c51901a7aa3199a873e\": rpc error: code = NotFound desc = could not find container \"8af92a74f303bc7baf7b2a8348f9bcdd3ffbf86176478c51901a7aa3199a873e\": container with ID starting with 8af92a74f303bc7baf7b2a8348f9bcdd3ffbf86176478c51901a7aa3199a873e not found: ID does not exist"
Sep 10 21:19:56.437590 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.437572    2580 scope.go:117] "RemoveContainer" containerID="9af8978930a9b72e17759d22a9d21b776b61cc139b534c3c0d6e940abe5e7cc6"
Sep 10 21:19:56.445935 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.445917    2580 scope.go:117] "RemoveContainer" containerID="9af8978930a9b72e17759d22a9d21b776b61cc139b534c3c0d6e940abe5e7cc6"
Sep 10 21:19:56.446226 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:19:56.446204    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9af8978930a9b72e17759d22a9d21b776b61cc139b534c3c0d6e940abe5e7cc6\": container with ID starting with 9af8978930a9b72e17759d22a9d21b776b61cc139b534c3c0d6e940abe5e7cc6 not found: ID does not exist" containerID="9af8978930a9b72e17759d22a9d21b776b61cc139b534c3c0d6e940abe5e7cc6"
Sep 10 21:19:56.446286 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.446232    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9af8978930a9b72e17759d22a9d21b776b61cc139b534c3c0d6e940abe5e7cc6"} err="failed to get container status \"9af8978930a9b72e17759d22a9d21b776b61cc139b534c3c0d6e940abe5e7cc6\": rpc error: code = NotFound desc = could not find container \"9af8978930a9b72e17759d22a9d21b776b61cc139b534c3c0d6e940abe5e7cc6\": container with ID starting with 9af8978930a9b72e17759d22a9d21b776b61cc139b534c3c0d6e940abe5e7cc6 not found: ID does not exist"
Sep 10 21:19:56.454684 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.454659    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"]
Sep 10 21:19:56.458865 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.458843    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-7f376d02-5c448f48b9-xmpxf"]
Sep 10 21:19:56.472035 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.472012    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"]
Sep 10 21:19:56.475375 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.475352    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7f376d02-f65959dcb-c4nzf"]
Sep 10 21:19:56.956881 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.956848    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1e1186d-fb7c-483a-bf39-88d90c57e821" path="/var/lib/kubelet/pods/a1e1186d-fb7c-483a-bf39-88d90c57e821/volumes"
Sep 10 21:19:56.957292 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:56.957256    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd7c48cc-79b3-4301-90d2-71e948271f65" path="/var/lib/kubelet/pods/fd7c48cc-79b3-4301-90d2-71e948271f65/volumes"
Sep 10 21:19:59.319486 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:19:59.319449    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:04.345372 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:04.345329    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:04.936587 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:04.936499    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:20:18.403706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:18.403674    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:20:24.691919 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:24.691878    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:24.747376 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:24.747347    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:32.040912 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:32.040873    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:20:40.913199 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:40.913167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:41.860952 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:41.860913    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:45.440513 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:45.440477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:20:47.961573 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:47.961530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:48.017346 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:48.017315    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:48.234292 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:48.234179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:49.542355 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:49.542254    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:49.633636 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:49.633604    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:49.734515 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:49.734459    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:50.297204 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:50.297162    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:50.393409 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:50.393357    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:50.494156 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:50.494121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:50.548435 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:50.548349    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:51.370631 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:51.370603    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:51.917144 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:51.917106    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:56.529933 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:56.529892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:58.803047 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:58.803013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:58.876030 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:58.875998    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:59.548438 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:20:59.548408    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:04.256054 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:04.256020    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:21:06.848337 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:06.848304    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:06.907234 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:06.907201    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:07.004352 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:07.004320    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:07.093191 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:07.093158    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:17.605203 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:17.605170    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:17.835140 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:17.835088    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:27.104268 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:27.104233    2580 scope.go:117] "RemoveContainer" containerID="af9e29650ebd23dd409d88b74363ffd36ef4acc48b01b2945a58dcc8e08e7806"
Sep 10 21:21:27.114899 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:27.114882    2580 scope.go:117] "RemoveContainer" containerID="d3877def28ce8e97b8c751c58e4c7929d31a412489326ec7d9de2190b1ac8960"
Sep 10 21:21:34.595902 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:34.595865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:38.827280 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:38.827244    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:38.944819 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:38.944773    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:21:44.990903 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:44.990865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:45.049714 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:45.049681    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:45.366723 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:45.366688    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:45.698250 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:45.698172    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:46.271375 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:46.271346    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:48.514351 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:48.514317    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:48.579971 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:48.579923    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:49.438333 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:49.438296    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:49.781286 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:49.781202    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:49.881763 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:49.881725    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:49.981851 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:49.981818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:50.261075 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.261027    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:50.348139 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.348103    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:50.393247 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.393218    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"]
Sep 10 21:21:50.393654 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.393642    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd7c48cc-79b3-4301-90d2-71e948271f65" containerName="maas-api"
Sep 10 21:21:50.393699 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.393656    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd7c48cc-79b3-4301-90d2-71e948271f65" containerName="maas-api"
Sep 10 21:21:50.393699 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.393674    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1e1186d-fb7c-483a-bf39-88d90c57e821" containerName="payload-processing"
Sep 10 21:21:50.393699 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.393680    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1e1186d-fb7c-483a-bf39-88d90c57e821" containerName="payload-processing"
Sep 10 21:21:50.393802 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.393744    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1e1186d-fb7c-483a-bf39-88d90c57e821" containerName="payload-processing"
Sep 10 21:21:50.393802 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.393753    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd7c48cc-79b3-4301-90d2-71e948271f65" containerName="maas-api"
Sep 10 21:21:50.398432 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.398416    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"
Sep 10 21:21:50.401404 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.401381    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-a249a2\""
Sep 10 21:21:50.401540 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.401433    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-a249a2-dockercfg-9mmh6\""
Sep 10 21:21:50.408157 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.408133    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"]
Sep 10 21:21:50.435934 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.435906    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"]
Sep 10 21:21:50.439584 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.439562    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"
Sep 10 21:21:50.452105 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.452077    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"]
Sep 10 21:21:50.550869 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.550841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e3a8227-ba8f-4db5-9adb-f42a584cdeb2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p\" (UID: \"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2\") " pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"
Sep 10 21:21:50.551022 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.550887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0793b060-b2e3-41ea-b795-86fa6bde743f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv\" (UID: \"0793b060-b2e3-41ea-b795-86fa6bde743f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"
Sep 10 21:21:50.551022 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.550971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9gh5\" (UniqueName: \"kubernetes.io/projected/0793b060-b2e3-41ea-b795-86fa6bde743f-kube-api-access-v9gh5\") pod \"payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv\" (UID: \"0793b060-b2e3-41ea-b795-86fa6bde743f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"
Sep 10 21:21:50.551022 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.551017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt9vf\" (UniqueName: \"kubernetes.io/projected/2e3a8227-ba8f-4db5-9adb-f42a584cdeb2-kube-api-access-kt9vf\") pod \"payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p\" (UID: \"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2\") " pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"
Sep 10 21:21:50.651840 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.651801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e3a8227-ba8f-4db5-9adb-f42a584cdeb2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p\" (UID: \"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2\") " pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"
Sep 10 21:21:50.652015 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.651858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0793b060-b2e3-41ea-b795-86fa6bde743f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv\" (UID: \"0793b060-b2e3-41ea-b795-86fa6bde743f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"
Sep 10 21:21:50.652015 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.651914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9gh5\" (UniqueName: \"kubernetes.io/projected/0793b060-b2e3-41ea-b795-86fa6bde743f-kube-api-access-v9gh5\") pod \"payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv\" (UID: \"0793b060-b2e3-41ea-b795-86fa6bde743f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"
Sep 10 21:21:50.652171 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.652035    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt9vf\" (UniqueName: \"kubernetes.io/projected/2e3a8227-ba8f-4db5-9adb-f42a584cdeb2-kube-api-access-kt9vf\") pod \"payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p\" (UID: \"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2\") " pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"
Sep 10 21:21:50.652663 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.652644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0793b060-b2e3-41ea-b795-86fa6bde743f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv\" (UID: \"0793b060-b2e3-41ea-b795-86fa6bde743f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"
Sep 10 21:21:50.652817 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.652800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e3a8227-ba8f-4db5-9adb-f42a584cdeb2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p\" (UID: \"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2\") " pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"
Sep 10 21:21:50.660566 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.660544    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9gh5\" (UniqueName: \"kubernetes.io/projected/0793b060-b2e3-41ea-b795-86fa6bde743f-kube-api-access-v9gh5\") pod \"payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv\" (UID: \"0793b060-b2e3-41ea-b795-86fa6bde743f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"
Sep 10 21:21:50.660688 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.660552    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt9vf\" (UniqueName: \"kubernetes.io/projected/2e3a8227-ba8f-4db5-9adb-f42a584cdeb2-kube-api-access-kt9vf\") pod \"payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p\" (UID: \"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2\") " pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"
Sep 10 21:21:50.710556 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.710534    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"
Sep 10 21:21:50.749695 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.749633    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"
Sep 10 21:21:50.990266 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:50.990197    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"]
Sep 10 21:21:51.000679 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:21:51.000653    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e3a8227_ba8f_4db5_9adb_f42a584cdeb2.slice/crio-660f8e59c35c2470e588452a5077af955c6318c6fa2836e88adfd08d57e5a346 WatchSource:0}: Error finding container 660f8e59c35c2470e588452a5077af955c6318c6fa2836e88adfd08d57e5a346: Status 404 returned error can't find the container with id 660f8e59c35c2470e588452a5077af955c6318c6fa2836e88adfd08d57e5a346
Sep 10 21:21:51.031779 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:51.031755    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"]
Sep 10 21:21:51.038204 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:21:51.038175    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0793b060_b2e3_41ea_b795_86fa6bde743f.slice/crio-77332f04cf595ed14f19c012e01eb8ab8c9cb48a1eaf358636422e784d68801b WatchSource:0}: Error finding container 77332f04cf595ed14f19c012e01eb8ab8c9cb48a1eaf358636422e784d68801b: Status 404 returned error can't find the container with id 77332f04cf595ed14f19c012e01eb8ab8c9cb48a1eaf358636422e784d68801b
Sep 10 21:21:51.059035 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:51.059002    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p" event={"ID":"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2","Type":"ContainerStarted","Data":"660f8e59c35c2470e588452a5077af955c6318c6fa2836e88adfd08d57e5a346"}
Sep 10 21:21:51.060466 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:51.060445    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv" event={"ID":"0793b060-b2e3-41ea-b795-86fa6bde743f","Type":"ContainerStarted","Data":"77332f04cf595ed14f19c012e01eb8ab8c9cb48a1eaf358636422e784d68801b"}
Sep 10 21:21:52.068120 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:52.068088    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv" event={"ID":"0793b060-b2e3-41ea-b795-86fa6bde743f","Type":"ContainerStarted","Data":"9cbb2a4bd87ef69ac4347748e5d6e80d662bba95a29c106db360148ab2eebbc1"}
Sep 10 21:21:52.068601 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:52.068176    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"
Sep 10 21:21:52.069774 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:52.069753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p" event={"ID":"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2","Type":"ContainerStarted","Data":"0cf93013509ed7d8222f76a184c6b308842d9bfdcd33e099eccac098aa5c53e4"}
Sep 10 21:21:52.069898 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:52.069861    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"
Sep 10 21:21:52.087254 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:52.087212    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv" podStartSLOduration=2.087198688 podStartE2EDuration="2.087198688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:21:52.08489868 +0000 UTC m=+1405.764397531" watchObservedRunningTime="2026-09-10 21:21:52.087198688 +0000 UTC m=+1405.766697563"
Sep 10 21:21:52.101273 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:52.101233    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p" podStartSLOduration=2.101219657 podStartE2EDuration="2.101219657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:21:52.100423526 +0000 UTC m=+1405.779922371" watchObservedRunningTime="2026-09-10 21:21:52.101219657 +0000 UTC m=+1405.780718505"
Sep 10 21:21:57.143962 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:21:57.143928    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:22:03.077573 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:03.077539    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"
Sep 10 21:22:03.078045 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:03.077874    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"
Sep 10 21:22:10.040389 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:10.040353    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:22:13.365826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:13.365792    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:13.579588 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:13.579544    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:14.069431 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:14.069402    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:14.412575 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:14.412499    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:14.508538 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:14.508494    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:17.645896 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:17.645865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:17.745585 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:17.745554    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:18.944649 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:18.944606    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"]
Sep 10 21:22:18.948535 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:18.948513    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:18.952780 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:18.951715    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-cf7171-serving-cert\""
Sep 10 21:22:18.959963 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:18.959893    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"]
Sep 10 21:22:18.982066 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:18.982042    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:19.007452 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.007427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ef7b186-8af2-4095-8765-3dbe9d2c9937-metrics-certs\") pod \"maas-api-e2e-api-b-cf7171-b48476d78-vzc7p\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:19.007571 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.007456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef7b186-8af2-4095-8765-3dbe9d2c9937-openshift-service-ca\") pod \"maas-api-e2e-api-b-cf7171-b48476d78-vzc7p\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:19.007571 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.007525    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w56g9\" (UniqueName: \"kubernetes.io/projected/2ef7b186-8af2-4095-8765-3dbe9d2c9937-kube-api-access-w56g9\") pod \"maas-api-e2e-api-b-cf7171-b48476d78-vzc7p\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:19.007571 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.007567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ef7b186-8af2-4095-8765-3dbe9d2c9937-maas-api-tls\") pod \"maas-api-e2e-api-b-cf7171-b48476d78-vzc7p\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:19.083288 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.083260    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:19.108627 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.108604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w56g9\" (UniqueName: \"kubernetes.io/projected/2ef7b186-8af2-4095-8765-3dbe9d2c9937-kube-api-access-w56g9\") pod \"maas-api-e2e-api-b-cf7171-b48476d78-vzc7p\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:19.108745 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.108666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ef7b186-8af2-4095-8765-3dbe9d2c9937-maas-api-tls\") pod \"maas-api-e2e-api-b-cf7171-b48476d78-vzc7p\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:19.108812 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.108768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ef7b186-8af2-4095-8765-3dbe9d2c9937-metrics-certs\") pod \"maas-api-e2e-api-b-cf7171-b48476d78-vzc7p\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:19.108812 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.108795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef7b186-8af2-4095-8765-3dbe9d2c9937-openshift-service-ca\") pod \"maas-api-e2e-api-b-cf7171-b48476d78-vzc7p\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:19.109559 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.109534    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef7b186-8af2-4095-8765-3dbe9d2c9937-openshift-service-ca\") pod \"maas-api-e2e-api-b-cf7171-b48476d78-vzc7p\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:19.111032 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.111015    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ef7b186-8af2-4095-8765-3dbe9d2c9937-maas-api-tls\") pod \"maas-api-e2e-api-b-cf7171-b48476d78-vzc7p\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:19.111186 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.111166    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ef7b186-8af2-4095-8765-3dbe9d2c9937-metrics-certs\") pod \"maas-api-e2e-api-b-cf7171-b48476d78-vzc7p\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:19.116908 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.116887    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w56g9\" (UniqueName: \"kubernetes.io/projected/2ef7b186-8af2-4095-8765-3dbe9d2c9937-kube-api-access-w56g9\") pod \"maas-api-e2e-api-b-cf7171-b48476d78-vzc7p\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:19.183572 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.183535    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:19.265342 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.265275    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:19.494401 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.494369    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:19.548295 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.548272    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"]
Sep 10 21:22:19.550731 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:22:19.550700    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ef7b186_8af2_4095_8765_3dbe9d2c9937.slice/crio-96ba1dddd7c400eacdfbf07888cd76a031107eb08fdff60dc481e16c4501f458 WatchSource:0}: Error finding container 96ba1dddd7c400eacdfbf07888cd76a031107eb08fdff60dc481e16c4501f458: Status 404 returned error can't find the container with id 96ba1dddd7c400eacdfbf07888cd76a031107eb08fdff60dc481e16c4501f458
Sep 10 21:22:19.552112 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.552092    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:22:19.591485 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:19.591459    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:20.226709 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:20.226631    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p" event={"ID":"2ef7b186-8af2-4095-8765-3dbe9d2c9937","Type":"ContainerStarted","Data":"910f6da60d738c59290a5c2aad2d9075c14975c2fbbcce64c570ceb9f7e98f29"}
Sep 10 21:22:20.226709 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:20.226693    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p" event={"ID":"2ef7b186-8af2-4095-8765-3dbe9d2c9937","Type":"ContainerStarted","Data":"96ba1dddd7c400eacdfbf07888cd76a031107eb08fdff60dc481e16c4501f458"}
Sep 10 21:22:20.227224 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:20.226840    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:20.246403 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:20.246353    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p" podStartSLOduration=1.920269674 podStartE2EDuration="2.246339601s" totalImagesPullingTime="326.069927ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:22:20.243989832 +0000 UTC m=+1433.923488682" watchObservedRunningTime="2026-09-10 21:22:20.246339601 +0000 UTC m=+1433.925838451"
Sep 10 21:22:26.238442 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:26.238412    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:22:27.136841 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:27.136803    2580 scope.go:117] "RemoveContainer" containerID="1d73fbffc647f02b1dbf14079d79db6e826caf0f943d71b14d706cd6ad897e2c"
Sep 10 21:22:28.047232 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:28.047188    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:22:36.578116 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:36.578080    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:37.280703 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:37.280648    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:41.391889 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:41.391856    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:42.245941 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:42.245910    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:22:42.665191 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:42.665156    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:45.904416 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:45.904371    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:47.071631 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:47.071599    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:47.292634 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:47.292602    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:48.013701 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:48.013664    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:48.112380 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:48.112346    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:54.393783 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:54.393750    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:22:55.986996 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:22:55.986960    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:01.702360 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:01.702313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:01.728629 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:01.728596    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"]
Sep 10 21:23:01.728851 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:01.728803    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p" podUID="2e3a8227-ba8f-4db5-9adb-f42a584cdeb2" containerName="payload-processing" containerID="cri-o://0cf93013509ed7d8222f76a184c6b308842d9bfdcd33e099eccac098aa5c53e4" gracePeriod=30
Sep 10 21:23:01.737323 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:01.737299    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"]
Sep 10 21:23:01.737510 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:01.737489    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv" podUID="0793b060-b2e3-41ea-b795-86fa6bde743f" containerName="payload-pre-processing" containerID="cri-o://9cbb2a4bd87ef69ac4347748e5d6e80d662bba95a29c106db360148ab2eebbc1" gracePeriod=30
Sep 10 21:23:01.777499 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:01.777470    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:01.845169 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:01.845141    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:01.954583 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:01.954475    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:01.991408 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:01.991384    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"
Sep 10 21:23:02.022195 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.022173    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"
Sep 10 21:23:02.039283 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.039250    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:02.100992 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.100728    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e3a8227-ba8f-4db5-9adb-f42a584cdeb2-plugins-config-volume\") pod \"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2\" (UID: \"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2\") "
Sep 10 21:23:02.100992 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.100825    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9gh5\" (UniqueName: \"kubernetes.io/projected/0793b060-b2e3-41ea-b795-86fa6bde743f-kube-api-access-v9gh5\") pod \"0793b060-b2e3-41ea-b795-86fa6bde743f\" (UID: \"0793b060-b2e3-41ea-b795-86fa6bde743f\") "
Sep 10 21:23:02.100992 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.100852    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kt9vf\" (UniqueName: \"kubernetes.io/projected/2e3a8227-ba8f-4db5-9adb-f42a584cdeb2-kube-api-access-kt9vf\") pod \"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2\" (UID: \"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2\") "
Sep 10 21:23:02.100992 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.100949    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0793b060-b2e3-41ea-b795-86fa6bde743f-plugins-config-volume\") pod \"0793b060-b2e3-41ea-b795-86fa6bde743f\" (UID: \"0793b060-b2e3-41ea-b795-86fa6bde743f\") "
Sep 10 21:23:02.101906 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.101870    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e3a8227-ba8f-4db5-9adb-f42a584cdeb2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2e3a8227-ba8f-4db5-9adb-f42a584cdeb2" (UID: "2e3a8227-ba8f-4db5-9adb-f42a584cdeb2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:23:02.102044 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.101900    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0793b060-b2e3-41ea-b795-86fa6bde743f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0793b060-b2e3-41ea-b795-86fa6bde743f" (UID: "0793b060-b2e3-41ea-b795-86fa6bde743f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:23:02.103705 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.103680    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0793b060-b2e3-41ea-b795-86fa6bde743f-kube-api-access-v9gh5" (OuterVolumeSpecName: "kube-api-access-v9gh5") pod "0793b060-b2e3-41ea-b795-86fa6bde743f" (UID: "0793b060-b2e3-41ea-b795-86fa6bde743f"). InnerVolumeSpecName "kube-api-access-v9gh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:23:02.103778 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.103708    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e3a8227-ba8f-4db5-9adb-f42a584cdeb2-kube-api-access-kt9vf" (OuterVolumeSpecName: "kube-api-access-kt9vf") pod "2e3a8227-ba8f-4db5-9adb-f42a584cdeb2" (UID: "2e3a8227-ba8f-4db5-9adb-f42a584cdeb2"). InnerVolumeSpecName "kube-api-access-kt9vf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:23:02.202224 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.202184    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9gh5\" (UniqueName: \"kubernetes.io/projected/0793b060-b2e3-41ea-b795-86fa6bde743f-kube-api-access-v9gh5\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:02.202224 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.202220    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt9vf\" (UniqueName: \"kubernetes.io/projected/2e3a8227-ba8f-4db5-9adb-f42a584cdeb2-kube-api-access-kt9vf\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:02.202224 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.202229    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0793b060-b2e3-41ea-b795-86fa6bde743f-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:02.202466 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.202238    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e3a8227-ba8f-4db5-9adb-f42a584cdeb2-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:02.354307 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.354266    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:02.451697 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.451656    2580 generic.go:356] "Generic (PLEG): container finished" podID="2e3a8227-ba8f-4db5-9adb-f42a584cdeb2" containerID="0cf93013509ed7d8222f76a184c6b308842d9bfdcd33e099eccac098aa5c53e4" exitCode=0
Sep 10 21:23:02.451855 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.451707    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"
Sep 10 21:23:02.451855 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.451742    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p" event={"ID":"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2","Type":"ContainerDied","Data":"0cf93013509ed7d8222f76a184c6b308842d9bfdcd33e099eccac098aa5c53e4"}
Sep 10 21:23:02.451855 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.451799    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p" event={"ID":"2e3a8227-ba8f-4db5-9adb-f42a584cdeb2","Type":"ContainerDied","Data":"660f8e59c35c2470e588452a5077af955c6318c6fa2836e88adfd08d57e5a346"}
Sep 10 21:23:02.451855 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.451827    2580 scope.go:117] "RemoveContainer" containerID="0cf93013509ed7d8222f76a184c6b308842d9bfdcd33e099eccac098aa5c53e4"
Sep 10 21:23:02.453362 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.453336    2580 generic.go:356] "Generic (PLEG): container finished" podID="0793b060-b2e3-41ea-b795-86fa6bde743f" containerID="9cbb2a4bd87ef69ac4347748e5d6e80d662bba95a29c106db360148ab2eebbc1" exitCode=0
Sep 10 21:23:02.453477 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.453384    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"
Sep 10 21:23:02.453477 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.453392    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv" event={"ID":"0793b060-b2e3-41ea-b795-86fa6bde743f","Type":"ContainerDied","Data":"9cbb2a4bd87ef69ac4347748e5d6e80d662bba95a29c106db360148ab2eebbc1"}
Sep 10 21:23:02.453477 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.453434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv" event={"ID":"0793b060-b2e3-41ea-b795-86fa6bde743f","Type":"ContainerDied","Data":"77332f04cf595ed14f19c012e01eb8ab8c9cb48a1eaf358636422e784d68801b"}
Sep 10 21:23:02.461071 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.461021    2580 scope.go:117] "RemoveContainer" containerID="0cf93013509ed7d8222f76a184c6b308842d9bfdcd33e099eccac098aa5c53e4"
Sep 10 21:23:02.461435 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:23:02.461414    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cf93013509ed7d8222f76a184c6b308842d9bfdcd33e099eccac098aa5c53e4\": container with ID starting with 0cf93013509ed7d8222f76a184c6b308842d9bfdcd33e099eccac098aa5c53e4 not found: ID does not exist" containerID="0cf93013509ed7d8222f76a184c6b308842d9bfdcd33e099eccac098aa5c53e4"
Sep 10 21:23:02.461530 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.461443    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cf93013509ed7d8222f76a184c6b308842d9bfdcd33e099eccac098aa5c53e4"} err="failed to get container status \"0cf93013509ed7d8222f76a184c6b308842d9bfdcd33e099eccac098aa5c53e4\": rpc error: code = NotFound desc = could not find container \"0cf93013509ed7d8222f76a184c6b308842d9bfdcd33e099eccac098aa5c53e4\": container with ID starting with 0cf93013509ed7d8222f76a184c6b308842d9bfdcd33e099eccac098aa5c53e4 not found: ID does not exist"
Sep 10 21:23:02.461530 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.461459    2580 scope.go:117] "RemoveContainer" containerID="9cbb2a4bd87ef69ac4347748e5d6e80d662bba95a29c106db360148ab2eebbc1"
Sep 10 21:23:02.470570 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.470545    2580 scope.go:117] "RemoveContainer" containerID="9cbb2a4bd87ef69ac4347748e5d6e80d662bba95a29c106db360148ab2eebbc1"
Sep 10 21:23:02.470851 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:23:02.470834    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cbb2a4bd87ef69ac4347748e5d6e80d662bba95a29c106db360148ab2eebbc1\": container with ID starting with 9cbb2a4bd87ef69ac4347748e5d6e80d662bba95a29c106db360148ab2eebbc1 not found: ID does not exist" containerID="9cbb2a4bd87ef69ac4347748e5d6e80d662bba95a29c106db360148ab2eebbc1"
Sep 10 21:23:02.470898 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.470859    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cbb2a4bd87ef69ac4347748e5d6e80d662bba95a29c106db360148ab2eebbc1"} err="failed to get container status \"9cbb2a4bd87ef69ac4347748e5d6e80d662bba95a29c106db360148ab2eebbc1\": rpc error: code = NotFound desc = could not find container \"9cbb2a4bd87ef69ac4347748e5d6e80d662bba95a29c106db360148ab2eebbc1\": container with ID starting with 9cbb2a4bd87ef69ac4347748e5d6e80d662bba95a29c106db360148ab2eebbc1 not found: ID does not exist"
Sep 10 21:23:02.488526 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.488498    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"]
Sep 10 21:23:02.492910 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.492885    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-a249a2-999d44f85-ztj6p"]
Sep 10 21:23:02.504601 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.504569    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"]
Sep 10 21:23:02.511756 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.511715    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-a249a2-68c5cc47cd-25bzv"]
Sep 10 21:23:02.957036 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.957003    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0793b060-b2e3-41ea-b795-86fa6bde743f" path="/var/lib/kubelet/pods/0793b060-b2e3-41ea-b795-86fa6bde743f/volumes"
Sep 10 21:23:02.957787 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:02.957767    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e3a8227-ba8f-4db5-9adb-f42a584cdeb2" path="/var/lib/kubelet/pods/2e3a8227-ba8f-4db5-9adb-f42a584cdeb2/volumes"
Sep 10 21:23:05.797881 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:05.797839    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:07.248331 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:07.248284    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:23:08.623369 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:08.623335    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:08.723503 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:08.723467    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:08.922285 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:08.922196    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:09.527805 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:09.527756    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:13.693789 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:13.693749    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:19.455134 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:19.455100    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:21.340575 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:21.340538    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:23:25.046103 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.046051    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"]
Sep 10 21:23:25.046891 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.046854    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p" podUID="2ef7b186-8af2-4095-8765-3dbe9d2c9937" containerName="maas-api" containerID="cri-o://910f6da60d738c59290a5c2aad2d9075c14975c2fbbcce64c570ceb9f7e98f29" gracePeriod=30
Sep 10 21:23:25.074206 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.074170    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:25.137587 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.137546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:25.198836 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.198797    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:25.296468 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.296404    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:23:25.302946 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.302919    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:25.321823 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.321795    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ef7b186-8af2-4095-8765-3dbe9d2c9937-maas-api-tls\") pod \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") "
Sep 10 21:23:25.322001 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.321891    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w56g9\" (UniqueName: \"kubernetes.io/projected/2ef7b186-8af2-4095-8765-3dbe9d2c9937-kube-api-access-w56g9\") pod \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") "
Sep 10 21:23:25.322001 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.321952    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef7b186-8af2-4095-8765-3dbe9d2c9937-openshift-service-ca\") pod \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") "
Sep 10 21:23:25.322151 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.322080    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ef7b186-8af2-4095-8765-3dbe9d2c9937-metrics-certs\") pod \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\" (UID: \"2ef7b186-8af2-4095-8765-3dbe9d2c9937\") "
Sep 10 21:23:25.322383 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.322348    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ef7b186-8af2-4095-8765-3dbe9d2c9937-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2ef7b186-8af2-4095-8765-3dbe9d2c9937" (UID: "2ef7b186-8af2-4095-8765-3dbe9d2c9937"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:23:25.324290 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.324252    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ef7b186-8af2-4095-8765-3dbe9d2c9937-kube-api-access-w56g9" (OuterVolumeSpecName: "kube-api-access-w56g9") pod "2ef7b186-8af2-4095-8765-3dbe9d2c9937" (UID: "2ef7b186-8af2-4095-8765-3dbe9d2c9937"). InnerVolumeSpecName "kube-api-access-w56g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:23:25.324592 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.324552    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ef7b186-8af2-4095-8765-3dbe9d2c9937-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2ef7b186-8af2-4095-8765-3dbe9d2c9937" (UID: "2ef7b186-8af2-4095-8765-3dbe9d2c9937"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:23:25.324693 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.324636    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ef7b186-8af2-4095-8765-3dbe9d2c9937-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2ef7b186-8af2-4095-8765-3dbe9d2c9937" (UID: "2ef7b186-8af2-4095-8765-3dbe9d2c9937"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:23:25.398515 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.398482    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:25.423003 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.422967    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w56g9\" (UniqueName: \"kubernetes.io/projected/2ef7b186-8af2-4095-8765-3dbe9d2c9937-kube-api-access-w56g9\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:25.423003 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.422999    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ef7b186-8af2-4095-8765-3dbe9d2c9937-openshift-service-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:25.423003 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.423009    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ef7b186-8af2-4095-8765-3dbe9d2c9937-metrics-certs\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:25.423316 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.423020    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ef7b186-8af2-4095-8765-3dbe9d2c9937-maas-api-tls\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:25.594955 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.594857    2580 generic.go:356] "Generic (PLEG): container finished" podID="2ef7b186-8af2-4095-8765-3dbe9d2c9937" containerID="910f6da60d738c59290a5c2aad2d9075c14975c2fbbcce64c570ceb9f7e98f29" exitCode=0
Sep 10 21:23:25.594955 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.594909    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"
Sep 10 21:23:25.594955 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.594942    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p" event={"ID":"2ef7b186-8af2-4095-8765-3dbe9d2c9937","Type":"ContainerDied","Data":"910f6da60d738c59290a5c2aad2d9075c14975c2fbbcce64c570ceb9f7e98f29"}
Sep 10 21:23:25.595301 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.594995    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p" event={"ID":"2ef7b186-8af2-4095-8765-3dbe9d2c9937","Type":"ContainerDied","Data":"96ba1dddd7c400eacdfbf07888cd76a031107eb08fdff60dc481e16c4501f458"}
Sep 10 21:23:25.595301 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.595020    2580 scope.go:117] "RemoveContainer" containerID="910f6da60d738c59290a5c2aad2d9075c14975c2fbbcce64c570ceb9f7e98f29"
Sep 10 21:23:25.604955 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.604937    2580 scope.go:117] "RemoveContainer" containerID="910f6da60d738c59290a5c2aad2d9075c14975c2fbbcce64c570ceb9f7e98f29"
Sep 10 21:23:25.605329 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:23:25.605306    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"910f6da60d738c59290a5c2aad2d9075c14975c2fbbcce64c570ceb9f7e98f29\": container with ID starting with 910f6da60d738c59290a5c2aad2d9075c14975c2fbbcce64c570ceb9f7e98f29 not found: ID does not exist" containerID="910f6da60d738c59290a5c2aad2d9075c14975c2fbbcce64c570ceb9f7e98f29"
Sep 10 21:23:25.605426 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.605337    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"910f6da60d738c59290a5c2aad2d9075c14975c2fbbcce64c570ceb9f7e98f29"} err="failed to get container status \"910f6da60d738c59290a5c2aad2d9075c14975c2fbbcce64c570ceb9f7e98f29\": rpc error: code = NotFound desc = could not find container \"910f6da60d738c59290a5c2aad2d9075c14975c2fbbcce64c570ceb9f7e98f29\": container with ID starting with 910f6da60d738c59290a5c2aad2d9075c14975c2fbbcce64c570ceb9f7e98f29 not found: ID does not exist"
Sep 10 21:23:25.619676 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.619639    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"]
Sep 10 21:23:25.621542 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.621531    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cf7171-b48476d78-vzc7p"]
Sep 10 21:23:25.679772 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:25.679740    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:26.957688 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:26.957656    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ef7b186-8af2-4095-8765-3dbe9d2c9937" path="/var/lib/kubelet/pods/2ef7b186-8af2-4095-8765-3dbe9d2c9937/volumes"
Sep 10 21:23:27.160463 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:27.160398    2580 scope.go:117] "RemoveContainer" containerID="106bfa3b795c6f9f2197a14968851103708d7912a57d8eac4e255f059fe08579"
Sep 10 21:23:32.185482 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:32.185414    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:32.285446 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:32.285406    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:32.450714 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:32.450601    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:33.067013 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:33.066964    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:37.440887 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:37.440850    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:23:40.122443 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:40.122375    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:43.379449 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.379412    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"]
Sep 10 21:23:43.379957 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.379659    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk" podUID="67da1ae0-b6a3-4e04-b5b4-23c0bbf830db" containerName="maas-api" containerID="cri-o://04c15f0fdd9c88dc031cc12810f6147e23e9768dc2bc2d2a9c8534ef8c2f0c83" gracePeriod=30
Sep 10 21:23:43.398146 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.398102    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:43.428859 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.428822    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"]
Sep 10 21:23:43.429153 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.429114    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb" podUID="cdf92600-aa61-4503-ba65-01a2d30e3ff3" containerName="payload-processing" containerID="cri-o://2e161306fa20fbc60c4fb218e93ed1bc5a56c536b656fff80c97e1dc777fc5cb" gracePeriod=30
Sep 10 21:23:43.498686 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.498653    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:43.599085 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.598283    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:43.661755 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.661632    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:43.695390 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.695350    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:23:43.698432 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.698407    2580 generic.go:356] "Generic (PLEG): container finished" podID="cdf92600-aa61-4503-ba65-01a2d30e3ff3" containerID="2e161306fa20fbc60c4fb218e93ed1bc5a56c536b656fff80c97e1dc777fc5cb" exitCode=0
Sep 10 21:23:43.698577 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.698483    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb" event={"ID":"cdf92600-aa61-4503-ba65-01a2d30e3ff3","Type":"ContainerDied","Data":"2e161306fa20fbc60c4fb218e93ed1bc5a56c536b656fff80c97e1dc777fc5cb"}
Sep 10 21:23:43.699918 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.699879    2580 generic.go:356] "Generic (PLEG): container finished" podID="67da1ae0-b6a3-4e04-b5b4-23c0bbf830db" containerID="04c15f0fdd9c88dc031cc12810f6147e23e9768dc2bc2d2a9c8534ef8c2f0c83" exitCode=0
Sep 10 21:23:43.700148 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.699935    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"
Sep 10 21:23:43.700148 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.699966    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk" event={"ID":"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db","Type":"ContainerDied","Data":"04c15f0fdd9c88dc031cc12810f6147e23e9768dc2bc2d2a9c8534ef8c2f0c83"}
Sep 10 21:23:43.700148 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.700013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk" event={"ID":"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db","Type":"ContainerDied","Data":"102875abce80b55acc8b5b730172096ac509ed70cc9799dd86c32ca8ed8613d4"}
Sep 10 21:23:43.700148 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.700041    2580 scope.go:117] "RemoveContainer" containerID="04c15f0fdd9c88dc031cc12810f6147e23e9768dc2bc2d2a9c8534ef8c2f0c83"
Sep 10 21:23:43.721666 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.721638    2580 scope.go:117] "RemoveContainer" containerID="04c15f0fdd9c88dc031cc12810f6147e23e9768dc2bc2d2a9c8534ef8c2f0c83"
Sep 10 21:23:43.721998 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:23:43.721970    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04c15f0fdd9c88dc031cc12810f6147e23e9768dc2bc2d2a9c8534ef8c2f0c83\": container with ID starting with 04c15f0fdd9c88dc031cc12810f6147e23e9768dc2bc2d2a9c8534ef8c2f0c83 not found: ID does not exist" containerID="04c15f0fdd9c88dc031cc12810f6147e23e9768dc2bc2d2a9c8534ef8c2f0c83"
Sep 10 21:23:43.722157 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.722010    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04c15f0fdd9c88dc031cc12810f6147e23e9768dc2bc2d2a9c8534ef8c2f0c83"} err="failed to get container status \"04c15f0fdd9c88dc031cc12810f6147e23e9768dc2bc2d2a9c8534ef8c2f0c83\": rpc error: code = NotFound desc = could not find container \"04c15f0fdd9c88dc031cc12810f6147e23e9768dc2bc2d2a9c8534ef8c2f0c83\": container with ID starting with 04c15f0fdd9c88dc031cc12810f6147e23e9768dc2bc2d2a9c8534ef8c2f0c83 not found: ID does not exist"
Sep 10 21:23:43.725690 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.725662    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"
Sep 10 21:23:43.805457 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.805423    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-metrics-certs\") pod \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") "
Sep 10 21:23:43.805619 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.805491    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-openshift-service-ca\") pod \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") "
Sep 10 21:23:43.805619 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.805557    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fptgn\" (UniqueName: \"kubernetes.io/projected/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-kube-api-access-fptgn\") pod \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") "
Sep 10 21:23:43.805619 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.805585    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-maas-api-tls\") pod \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\" (UID: \"67da1ae0-b6a3-4e04-b5b4-23c0bbf830db\") "
Sep 10 21:23:43.805619 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.805614    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96kx4\" (UniqueName: \"kubernetes.io/projected/cdf92600-aa61-4503-ba65-01a2d30e3ff3-kube-api-access-96kx4\") pod \"cdf92600-aa61-4503-ba65-01a2d30e3ff3\" (UID: \"cdf92600-aa61-4503-ba65-01a2d30e3ff3\") "
Sep 10 21:23:43.805844 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.805637    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdf92600-aa61-4503-ba65-01a2d30e3ff3-plugins-config-volume\") pod \"cdf92600-aa61-4503-ba65-01a2d30e3ff3\" (UID: \"cdf92600-aa61-4503-ba65-01a2d30e3ff3\") "
Sep 10 21:23:43.805905 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.805879    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "67da1ae0-b6a3-4e04-b5b4-23c0bbf830db" (UID: "67da1ae0-b6a3-4e04-b5b4-23c0bbf830db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:23:43.806027 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.805997    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-openshift-service-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:43.806319 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.806273    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdf92600-aa61-4503-ba65-01a2d30e3ff3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cdf92600-aa61-4503-ba65-01a2d30e3ff3" (UID: "cdf92600-aa61-4503-ba65-01a2d30e3ff3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:23:43.808015 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.807986    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf92600-aa61-4503-ba65-01a2d30e3ff3-kube-api-access-96kx4" (OuterVolumeSpecName: "kube-api-access-96kx4") pod "cdf92600-aa61-4503-ba65-01a2d30e3ff3" (UID: "cdf92600-aa61-4503-ba65-01a2d30e3ff3"). InnerVolumeSpecName "kube-api-access-96kx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:23:43.808436 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.808403    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-kube-api-access-fptgn" (OuterVolumeSpecName: "kube-api-access-fptgn") pod "67da1ae0-b6a3-4e04-b5b4-23c0bbf830db" (UID: "67da1ae0-b6a3-4e04-b5b4-23c0bbf830db"). InnerVolumeSpecName "kube-api-access-fptgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:23:43.808553 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.808438    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "67da1ae0-b6a3-4e04-b5b4-23c0bbf830db" (UID: "67da1ae0-b6a3-4e04-b5b4-23c0bbf830db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:23:43.808553 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.808458    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "67da1ae0-b6a3-4e04-b5b4-23c0bbf830db" (UID: "67da1ae0-b6a3-4e04-b5b4-23c0bbf830db"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:23:43.906861 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.906815    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fptgn\" (UniqueName: \"kubernetes.io/projected/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-kube-api-access-fptgn\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:43.906861 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.906849    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-maas-api-tls\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:43.906861 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.906859    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96kx4\" (UniqueName: \"kubernetes.io/projected/cdf92600-aa61-4503-ba65-01a2d30e3ff3-kube-api-access-96kx4\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:43.906861 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.906868    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdf92600-aa61-4503-ba65-01a2d30e3ff3-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:43.907269 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.906877    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db-metrics-certs\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:23:43.962154 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:43.961963    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:44.025628 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:44.025580    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"]
Sep 10 21:23:44.029133 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:44.029091    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-501c57-64d7647746-n7qsk"]
Sep 10 21:23:44.707204 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:44.707117    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb" event={"ID":"cdf92600-aa61-4503-ba65-01a2d30e3ff3","Type":"ContainerDied","Data":"aeb94ac72a4a0a0d9d292e26ab57780fbdc517a56a0635627b3bae523bcdaa77"}
Sep 10 21:23:44.707204 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:44.707189    2580 scope.go:117] "RemoveContainer" containerID="2e161306fa20fbc60c4fb218e93ed1bc5a56c536b656fff80c97e1dc777fc5cb"
Sep 10 21:23:44.707830 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:44.707132    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"
Sep 10 21:23:44.733013 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:44.732982    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"]
Sep 10 21:23:44.734952 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:44.734929    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-501c57-65655c86dc-mtqzb"]
Sep 10 21:23:44.956449 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:44.956407    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67da1ae0-b6a3-4e04-b5b4-23c0bbf830db" path="/var/lib/kubelet/pods/67da1ae0-b6a3-4e04-b5b4-23c0bbf830db/volumes"
Sep 10 21:23:44.956875 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:44.956854    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdf92600-aa61-4503-ba65-01a2d30e3ff3" path="/var/lib/kubelet/pods/cdf92600-aa61-4503-ba65-01a2d30e3ff3/volumes"
Sep 10 21:23:48.646815 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:48.646777    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:23:49.875027 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:49.874982    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:49.970264 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:49.970229    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:50.141145 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:50.141039    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:50.605895 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:50.605862    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:56.137309 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:56.137264    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:56.366576 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:56.366545    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:56.825576 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:56.825531    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:57.143008 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:57.142932    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:57.698395 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:23:57.698355    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:00.227816 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:00.227775    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:00.325363 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:00.325332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:00.443014 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:00.442975    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:24:01.011787 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.011759    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:01.050720 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.050673    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"]
Sep 10 21:24:01.050972 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.050942    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx" podUID="aa04180c-ace2-4253-8200-133d7db17b21" containerName="payload-pre-processing" containerID="cri-o://31f6438a599b883ec57296311682e45c6ba4ae3e5958976982a1e26c8a63dbfb" gracePeriod=30
Sep 10 21:24:01.072208 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.072178    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:01.128077 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.128037    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:01.204101 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.204032    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:01.254985 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.254945    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:01.319868 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.319838    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"
Sep 10 21:24:01.479120 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.479086    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa04180c-ace2-4253-8200-133d7db17b21-plugins-config-volume\") pod \"aa04180c-ace2-4253-8200-133d7db17b21\" (UID: \"aa04180c-ace2-4253-8200-133d7db17b21\") "
Sep 10 21:24:01.479325 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.479164    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xv6zn\" (UniqueName: \"kubernetes.io/projected/aa04180c-ace2-4253-8200-133d7db17b21-kube-api-access-xv6zn\") pod \"aa04180c-ace2-4253-8200-133d7db17b21\" (UID: \"aa04180c-ace2-4253-8200-133d7db17b21\") "
Sep 10 21:24:01.479541 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.479517    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa04180c-ace2-4253-8200-133d7db17b21-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aa04180c-ace2-4253-8200-133d7db17b21" (UID: "aa04180c-ace2-4253-8200-133d7db17b21"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:01.481256 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.481235    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa04180c-ace2-4253-8200-133d7db17b21-kube-api-access-xv6zn" (OuterVolumeSpecName: "kube-api-access-xv6zn") pod "aa04180c-ace2-4253-8200-133d7db17b21" (UID: "aa04180c-ace2-4253-8200-133d7db17b21"). InnerVolumeSpecName "kube-api-access-xv6zn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:01.536237 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.536182    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:01.580482 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.580389    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv6zn\" (UniqueName: \"kubernetes.io/projected/aa04180c-ace2-4253-8200-133d7db17b21-kube-api-access-xv6zn\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:24:01.580482 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.580419    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa04180c-ace2-4253-8200-133d7db17b21-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:24:01.634651 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.634615    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:01.802656 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.802615    2580 generic.go:356] "Generic (PLEG): container finished" podID="aa04180c-ace2-4253-8200-133d7db17b21" containerID="31f6438a599b883ec57296311682e45c6ba4ae3e5958976982a1e26c8a63dbfb" exitCode=0
Sep 10 21:24:01.802843 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.802684    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"
Sep 10 21:24:01.802843 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.802704    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx" event={"ID":"aa04180c-ace2-4253-8200-133d7db17b21","Type":"ContainerDied","Data":"31f6438a599b883ec57296311682e45c6ba4ae3e5958976982a1e26c8a63dbfb"}
Sep 10 21:24:01.802843 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.802783    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx" event={"ID":"aa04180c-ace2-4253-8200-133d7db17b21","Type":"ContainerDied","Data":"3b83cfadd3741fc8cd765d5eb30c626c1add83393dc72ff56c6ce185679ff856"}
Sep 10 21:24:01.802843 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.802810    2580 scope.go:117] "RemoveContainer" containerID="31f6438a599b883ec57296311682e45c6ba4ae3e5958976982a1e26c8a63dbfb"
Sep 10 21:24:01.812268 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.812250    2580 scope.go:117] "RemoveContainer" containerID="31f6438a599b883ec57296311682e45c6ba4ae3e5958976982a1e26c8a63dbfb"
Sep 10 21:24:01.812624 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:24:01.812600    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31f6438a599b883ec57296311682e45c6ba4ae3e5958976982a1e26c8a63dbfb\": container with ID starting with 31f6438a599b883ec57296311682e45c6ba4ae3e5958976982a1e26c8a63dbfb not found: ID does not exist" containerID="31f6438a599b883ec57296311682e45c6ba4ae3e5958976982a1e26c8a63dbfb"
Sep 10 21:24:01.812726 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.812633    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31f6438a599b883ec57296311682e45c6ba4ae3e5958976982a1e26c8a63dbfb"} err="failed to get container status \"31f6438a599b883ec57296311682e45c6ba4ae3e5958976982a1e26c8a63dbfb\": rpc error: code = NotFound desc = could not find container \"31f6438a599b883ec57296311682e45c6ba4ae3e5958976982a1e26c8a63dbfb\": container with ID starting with 31f6438a599b883ec57296311682e45c6ba4ae3e5958976982a1e26c8a63dbfb not found: ID does not exist"
Sep 10 21:24:01.831145 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.831046    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"]
Sep 10 21:24:01.837613 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.837586    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4ad710-5f4ddb4b6-gxdfx"]
Sep 10 21:24:01.868616 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:01.868583    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:02.006143 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.006111    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:02.104086 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.103968    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:02.108217 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108182    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"]
Sep 10 21:24:02.108597 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108586    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0793b060-b2e3-41ea-b795-86fa6bde743f" containerName="payload-pre-processing"
Sep 10 21:24:02.108642 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108599    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0793b060-b2e3-41ea-b795-86fa6bde743f" containerName="payload-pre-processing"
Sep 10 21:24:02.108642 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108606    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ef7b186-8af2-4095-8765-3dbe9d2c9937" containerName="maas-api"
Sep 10 21:24:02.108642 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108612    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ef7b186-8af2-4095-8765-3dbe9d2c9937" containerName="maas-api"
Sep 10 21:24:02.108642 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108625    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf92600-aa61-4503-ba65-01a2d30e3ff3" containerName="payload-processing"
Sep 10 21:24:02.108642 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108631    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf92600-aa61-4503-ba65-01a2d30e3ff3" containerName="payload-processing"
Sep 10 21:24:02.108642 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108639    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e3a8227-ba8f-4db5-9adb-f42a584cdeb2" containerName="payload-processing"
Sep 10 21:24:02.108826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108645    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e3a8227-ba8f-4db5-9adb-f42a584cdeb2" containerName="payload-processing"
Sep 10 21:24:02.108826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108656    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa04180c-ace2-4253-8200-133d7db17b21" containerName="payload-pre-processing"
Sep 10 21:24:02.108826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108661    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa04180c-ace2-4253-8200-133d7db17b21" containerName="payload-pre-processing"
Sep 10 21:24:02.108826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108674    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67da1ae0-b6a3-4e04-b5b4-23c0bbf830db" containerName="maas-api"
Sep 10 21:24:02.108826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108679    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="67da1ae0-b6a3-4e04-b5b4-23c0bbf830db" containerName="maas-api"
Sep 10 21:24:02.108826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108733    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e3a8227-ba8f-4db5-9adb-f42a584cdeb2" containerName="payload-processing"
Sep 10 21:24:02.108826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108741    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ef7b186-8af2-4095-8765-3dbe9d2c9937" containerName="maas-api"
Sep 10 21:24:02.108826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108748    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdf92600-aa61-4503-ba65-01a2d30e3ff3" containerName="payload-processing"
Sep 10 21:24:02.108826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108757    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="67da1ae0-b6a3-4e04-b5b4-23c0bbf830db" containerName="maas-api"
Sep 10 21:24:02.108826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108763    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0793b060-b2e3-41ea-b795-86fa6bde743f" containerName="payload-pre-processing"
Sep 10 21:24:02.108826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.108768    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa04180c-ace2-4253-8200-133d7db17b21" containerName="payload-pre-processing"
Sep 10 21:24:02.113185 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.113153    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"
Sep 10 21:24:02.115929 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.115909    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-3bbf7d-dockercfg-9vxnn\""
Sep 10 21:24:02.116035 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.115918    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-3bbf7d\""
Sep 10 21:24:02.125072 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.125035    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"]
Sep 10 21:24:02.289256 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.289215    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpzd2\" (UniqueName: \"kubernetes.io/projected/ff076e71-30bc-4693-9853-2eb0acc8be6b-kube-api-access-kpzd2\") pod \"payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9\" (UID: \"ff076e71-30bc-4693-9853-2eb0acc8be6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"
Sep 10 21:24:02.289685 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.289401    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff076e71-30bc-4693-9853-2eb0acc8be6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9\" (UID: \"ff076e71-30bc-4693-9853-2eb0acc8be6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"
Sep 10 21:24:02.390992 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.390899    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff076e71-30bc-4693-9853-2eb0acc8be6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9\" (UID: \"ff076e71-30bc-4693-9853-2eb0acc8be6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"
Sep 10 21:24:02.390992 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.390945    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpzd2\" (UniqueName: \"kubernetes.io/projected/ff076e71-30bc-4693-9853-2eb0acc8be6b-kube-api-access-kpzd2\") pod \"payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9\" (UID: \"ff076e71-30bc-4693-9853-2eb0acc8be6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"
Sep 10 21:24:02.391647 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.391624    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff076e71-30bc-4693-9853-2eb0acc8be6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9\" (UID: \"ff076e71-30bc-4693-9853-2eb0acc8be6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"
Sep 10 21:24:02.399644 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.399613    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpzd2\" (UniqueName: \"kubernetes.io/projected/ff076e71-30bc-4693-9853-2eb0acc8be6b-kube-api-access-kpzd2\") pod \"payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9\" (UID: \"ff076e71-30bc-4693-9853-2eb0acc8be6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"
Sep 10 21:24:02.424518 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.424477    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"
Sep 10 21:24:02.700172 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.700143    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"]
Sep 10 21:24:02.701738 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:24:02.701708    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff076e71_30bc_4693_9853_2eb0acc8be6b.slice/crio-c099f475b055d0fc83d8ffd7cc528c042fff62619f2bec8e7f5287a34724a690 WatchSource:0}: Error finding container c099f475b055d0fc83d8ffd7cc528c042fff62619f2bec8e7f5287a34724a690: Status 404 returned error can't find the container with id c099f475b055d0fc83d8ffd7cc528c042fff62619f2bec8e7f5287a34724a690
Sep 10 21:24:02.811082 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.811013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9" event={"ID":"ff076e71-30bc-4693-9853-2eb0acc8be6b","Type":"ContainerStarted","Data":"810e8515281f6ff6e50dd3c556c2d1e10d5697e9928c3bcf93719302c2e21b46"}
Sep 10 21:24:02.811274 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.811109    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9" event={"ID":"ff076e71-30bc-4693-9853-2eb0acc8be6b","Type":"ContainerStarted","Data":"c099f475b055d0fc83d8ffd7cc528c042fff62619f2bec8e7f5287a34724a690"}
Sep 10 21:24:02.811274 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.811148    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"
Sep 10 21:24:02.830169 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.830113    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9" podStartSLOduration=0.830095924 podStartE2EDuration="830.095924ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:24:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:24:02.828161744 +0000 UTC m=+1536.507660611" watchObservedRunningTime="2026-09-10 21:24:02.830095924 +0000 UTC m=+1536.509594773"
Sep 10 21:24:02.957976 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:02.957896    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa04180c-ace2-4253-8200-133d7db17b21" path="/var/lib/kubelet/pods/aa04180c-ace2-4253-8200-133d7db17b21/volumes"
Sep 10 21:24:06.071325 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:06.071282    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:08.408190 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:08.408144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:08.475990 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:08.475956    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:08.699774 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:08.699691    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:11.443643 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:11.443607    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:24:13.820646 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:13.820616    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"
Sep 10 21:24:22.516222 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:22.516169    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:25.274459 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:25.274405    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:25.979986 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:25.979936    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:26.314476 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:26.314440    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:26.516104 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:26.516050    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:29.413046 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:29.412992    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:29.507554 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:29.507515    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:30.542092 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.542041    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"]
Sep 10 21:24:30.546536 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.546504    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:30.549304 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.549282    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-a581b5-serving-cert\""
Sep 10 21:24:30.557650 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.557526    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"]
Sep 10 21:24:30.574429 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.574394    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:30.637767 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.637729    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:30.657041 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.657005    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-metrics-certs\") pod \"maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:30.657215 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.657158    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:30.657215 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.657189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g62gr\" (UniqueName: \"kubernetes.io/projected/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-kube-api-access-g62gr\") pod \"maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:30.657215 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.657210    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:30.739815 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.739782    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:30.758148 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.758110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:30.758319 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.758163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-metrics-certs\") pod \"maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:30.758319 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.758274    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:30.758319 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.758298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g62gr\" (UniqueName: \"kubernetes.io/projected/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-kube-api-access-g62gr\") pod \"maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:30.758876 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.758852    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:30.760627 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.760603    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-metrics-certs\") pod \"maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:30.760790 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.760771    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:30.767300 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.767281    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g62gr\" (UniqueName: \"kubernetes.io/projected/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-kube-api-access-g62gr\") pod \"maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:30.860939 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:30.860847    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:31.067691 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.067655    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:31.140755 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.140730    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"]
Sep 10 21:24:31.142749 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:24:31.142712    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6802adb0_289e_422e_b16f_4b2cc6a9a5b2.slice/crio-172f4d59ee3b1122f5c41c3b57347a9dedc7e5811bca0763dfeafd7814bc9fc0 WatchSource:0}: Error finding container 172f4d59ee3b1122f5c41c3b57347a9dedc7e5811bca0763dfeafd7814bc9fc0: Status 404 returned error can't find the container with id 172f4d59ee3b1122f5c41c3b57347a9dedc7e5811bca0763dfeafd7814bc9fc0
Sep 10 21:24:31.143899 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.143871    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:31.175784 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.175751    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"]
Sep 10 21:24:31.180598 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.180580    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"
Sep 10 21:24:31.183422 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.183339    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-a581b5-dockercfg-l9ks6\""
Sep 10 21:24:31.183564 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.183387    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-a581b5\""
Sep 10 21:24:31.191823 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.191798    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"]
Sep 10 21:24:31.262244 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.262196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt6tr\" (UniqueName: \"kubernetes.io/projected/c6768016-bca0-45aa-a732-1d4f44e2fd79-kube-api-access-zt6tr\") pod \"payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6\" (UID: \"c6768016-bca0-45aa-a732-1d4f44e2fd79\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"
Sep 10 21:24:31.262433 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.262401    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6768016-bca0-45aa-a732-1d4f44e2fd79-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6\" (UID: \"c6768016-bca0-45aa-a732-1d4f44e2fd79\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"
Sep 10 21:24:31.363908 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.363869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt6tr\" (UniqueName: \"kubernetes.io/projected/c6768016-bca0-45aa-a732-1d4f44e2fd79-kube-api-access-zt6tr\") pod \"payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6\" (UID: \"c6768016-bca0-45aa-a732-1d4f44e2fd79\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"
Sep 10 21:24:31.364118 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.363959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6768016-bca0-45aa-a732-1d4f44e2fd79-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6\" (UID: \"c6768016-bca0-45aa-a732-1d4f44e2fd79\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"
Sep 10 21:24:31.364679 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.364655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6768016-bca0-45aa-a732-1d4f44e2fd79-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6\" (UID: \"c6768016-bca0-45aa-a732-1d4f44e2fd79\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"
Sep 10 21:24:31.372905 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.372882    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt6tr\" (UniqueName: \"kubernetes.io/projected/c6768016-bca0-45aa-a732-1d4f44e2fd79-kube-api-access-zt6tr\") pod \"payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6\" (UID: \"c6768016-bca0-45aa-a732-1d4f44e2fd79\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"
Sep 10 21:24:31.492742 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.492650    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"
Sep 10 21:24:31.765899 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.765869    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"]
Sep 10 21:24:31.766646 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:24:31.766621    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6768016_bca0_45aa_a732_1d4f44e2fd79.slice/crio-c8c1d7adf810c25ee126d2f59b3ec8946e4adef61931de6a50cbd7b6c37c94d4 WatchSource:0}: Error finding container c8c1d7adf810c25ee126d2f59b3ec8946e4adef61931de6a50cbd7b6c37c94d4: Status 404 returned error can't find the container with id c8c1d7adf810c25ee126d2f59b3ec8946e4adef61931de6a50cbd7b6c37c94d4
Sep 10 21:24:31.971220 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.971186    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6" event={"ID":"c6768016-bca0-45aa-a732-1d4f44e2fd79","Type":"ContainerStarted","Data":"707fc7c183c9360c5dc53d5041c445e305f9c41ba3df916653358d81b60e0bcc"}
Sep 10 21:24:31.971424 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.971339    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"
Sep 10 21:24:31.971424 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.971377    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6" event={"ID":"c6768016-bca0-45aa-a732-1d4f44e2fd79","Type":"ContainerStarted","Data":"c8c1d7adf810c25ee126d2f59b3ec8946e4adef61931de6a50cbd7b6c37c94d4"}
Sep 10 21:24:31.972741 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.972720    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb" event={"ID":"6802adb0-289e-422e-b16f-4b2cc6a9a5b2","Type":"ContainerStarted","Data":"8177840284e5942357711e4f2b7f5a66639e4647c43dcc3c627a437f9e91de13"}
Sep 10 21:24:31.972849 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.972754    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb" event={"ID":"6802adb0-289e-422e-b16f-4b2cc6a9a5b2","Type":"ContainerStarted","Data":"172f4d59ee3b1122f5c41c3b57347a9dedc7e5811bca0763dfeafd7814bc9fc0"}
Sep 10 21:24:31.972849 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.972793    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:31.990795 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:31.990749    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6" podStartSLOduration=0.990734847 podStartE2EDuration="990.734847ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:24:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:24:31.987944326 +0000 UTC m=+1565.667443175" watchObservedRunningTime="2026-09-10 21:24:31.990734847 +0000 UTC m=+1565.670233696"
Sep 10 21:24:32.007238 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:32.007190    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb" podStartSLOduration=1.5759180640000001 podStartE2EDuration="2.007173904s" totalImagesPullingTime="431.25584ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:24:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:24:32.005676799 +0000 UTC m=+1565.685175651" watchObservedRunningTime="2026-09-10 21:24:32.007173904 +0000 UTC m=+1565.686672757"
Sep 10 21:24:36.847822 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:36.847787    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:24:37.983891 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:37.983858    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:24:42.980621 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:42.980586    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"
Sep 10 21:24:44.514755 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:44.514722    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:45.943518 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:45.943480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:54.470815 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:24:54.470755    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:11.703954 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:11.703917    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:15.535952 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:15.535917    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:25:29.136159 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:29.136052    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:25:36.084793 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:36.084758    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:36.420928 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:36.420839    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:36.516669 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:36.516621    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:39.792825 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:39.792788    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:39.891231 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:39.891198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:41.072913 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.072886    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:41.132479 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.132443    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:41.232540 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.232509    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:41.537354 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.537272    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:41.637664 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.637628    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:41.672020 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.671987    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"]
Sep 10 21:25:41.675790 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.675767    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"
Sep 10 21:25:41.678222 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.678204    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-d24c8a\""
Sep 10 21:25:41.678318 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.678232    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-d24c8a-dockercfg-r8z46\""
Sep 10 21:25:41.686540 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.686517    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"]
Sep 10 21:25:41.708741 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.708716    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n545b\" (UniqueName: \"kubernetes.io/projected/1438f80e-78d0-4689-b67b-e6554c512453-kube-api-access-n545b\") pod \"payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc\" (UID: \"1438f80e-78d0-4689-b67b-e6554c512453\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"
Sep 10 21:25:41.708884 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.708861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1438f80e-78d0-4689-b67b-e6554c512453-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc\" (UID: \"1438f80e-78d0-4689-b67b-e6554c512453\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"
Sep 10 21:25:41.740399 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.740371    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"]
Sep 10 21:25:41.743909 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.743893    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"
Sep 10 21:25:41.758288 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.758260    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"]
Sep 10 21:25:41.809812 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.809772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1438f80e-78d0-4689-b67b-e6554c512453-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc\" (UID: \"1438f80e-78d0-4689-b67b-e6554c512453\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"
Sep 10 21:25:41.810167 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.809834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdjdk\" (UniqueName: \"kubernetes.io/projected/98266168-b5f0-4542-8cce-f15eac2740e5-kube-api-access-zdjdk\") pod \"payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd\" (UID: \"98266168-b5f0-4542-8cce-f15eac2740e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"
Sep 10 21:25:41.810167 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.809876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n545b\" (UniqueName: \"kubernetes.io/projected/1438f80e-78d0-4689-b67b-e6554c512453-kube-api-access-n545b\") pod \"payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc\" (UID: \"1438f80e-78d0-4689-b67b-e6554c512453\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"
Sep 10 21:25:41.810296 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.810211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98266168-b5f0-4542-8cce-f15eac2740e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd\" (UID: \"98266168-b5f0-4542-8cce-f15eac2740e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"
Sep 10 21:25:41.810780 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.810758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1438f80e-78d0-4689-b67b-e6554c512453-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc\" (UID: \"1438f80e-78d0-4689-b67b-e6554c512453\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"
Sep 10 21:25:41.819426 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.819399    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n545b\" (UniqueName: \"kubernetes.io/projected/1438f80e-78d0-4689-b67b-e6554c512453-kube-api-access-n545b\") pod \"payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc\" (UID: \"1438f80e-78d0-4689-b67b-e6554c512453\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"
Sep 10 21:25:41.911407 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.911370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98266168-b5f0-4542-8cce-f15eac2740e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd\" (UID: \"98266168-b5f0-4542-8cce-f15eac2740e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"
Sep 10 21:25:41.911588 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.911427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdjdk\" (UniqueName: \"kubernetes.io/projected/98266168-b5f0-4542-8cce-f15eac2740e5-kube-api-access-zdjdk\") pod \"payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd\" (UID: \"98266168-b5f0-4542-8cce-f15eac2740e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"
Sep 10 21:25:41.912247 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.912227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98266168-b5f0-4542-8cce-f15eac2740e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd\" (UID: \"98266168-b5f0-4542-8cce-f15eac2740e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"
Sep 10 21:25:41.919789 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.919765    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdjdk\" (UniqueName: \"kubernetes.io/projected/98266168-b5f0-4542-8cce-f15eac2740e5-kube-api-access-zdjdk\") pod \"payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd\" (UID: \"98266168-b5f0-4542-8cce-f15eac2740e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"
Sep 10 21:25:41.987932 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:41.987903    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"
Sep 10 21:25:42.055280 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:42.054440    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"
Sep 10 21:25:42.266371 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:42.266342    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"]
Sep 10 21:25:42.268033 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:25:42.268011    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1438f80e_78d0_4689_b67b_e6554c512453.slice/crio-b28417b1566b575437246317c88558fb0c2a896ccd51977613aa88b904abe60a WatchSource:0}: Error finding container b28417b1566b575437246317c88558fb0c2a896ccd51977613aa88b904abe60a: Status 404 returned error can't find the container with id b28417b1566b575437246317c88558fb0c2a896ccd51977613aa88b904abe60a
Sep 10 21:25:42.336279 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:42.336247    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"]
Sep 10 21:25:42.338865 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:25:42.338839    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98266168_b5f0_4542_8cce_f15eac2740e5.slice/crio-e252e8274000bf0def9010fa769f8a558d99416d8da1b4ec79f1738496baf9fb WatchSource:0}: Error finding container e252e8274000bf0def9010fa769f8a558d99416d8da1b4ec79f1738496baf9fb: Status 404 returned error can't find the container with id e252e8274000bf0def9010fa769f8a558d99416d8da1b4ec79f1738496baf9fb
Sep 10 21:25:42.342815 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:42.342793    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc" event={"ID":"1438f80e-78d0-4689-b67b-e6554c512453","Type":"ContainerStarted","Data":"1cd4e37d61049946fd067bea647dca781801982e0870a40c7381e1e9188dc87e"}
Sep 10 21:25:42.342898 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:42.342842    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc" event={"ID":"1438f80e-78d0-4689-b67b-e6554c512453","Type":"ContainerStarted","Data":"b28417b1566b575437246317c88558fb0c2a896ccd51977613aa88b904abe60a"}
Sep 10 21:25:42.345785 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:42.345762    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"
Sep 10 21:25:42.362863 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:42.362815    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc" podStartSLOduration=1.36280019 podStartE2EDuration="1.36280019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:42.361049739 +0000 UTC m=+1636.040548588" watchObservedRunningTime="2026-09-10 21:25:42.36280019 +0000 UTC m=+1636.042299040"
Sep 10 21:25:43.350934 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:43.350895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd" event={"ID":"98266168-b5f0-4542-8cce-f15eac2740e5","Type":"ContainerStarted","Data":"63b79e555c413c70847e873630b585c9699b65944c2293d2d0f7914457dfbf94"}
Sep 10 21:25:43.351417 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:43.350951    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd" event={"ID":"98266168-b5f0-4542-8cce-f15eac2740e5","Type":"ContainerStarted","Data":"e252e8274000bf0def9010fa769f8a558d99416d8da1b4ec79f1738496baf9fb"}
Sep 10 21:25:44.356472 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:44.356436    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"
Sep 10 21:25:48.437464 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:48.437414    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd" podStartSLOduration=7.437397605 podStartE2EDuration="7.437397605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:43.370103349 +0000 UTC m=+1637.049602203" watchObservedRunningTime="2026-09-10 21:25:48.437397605 +0000 UTC m=+1642.116896454"
Sep 10 21:25:48.437827 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:48.437541    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:25:51.670026 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:51.669988    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:52.305182 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:52.305147    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:52.365553 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:52.365521    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:53.353165 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:53.353130    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"
Sep 10 21:25:55.364235 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:25:55.364204    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"
Sep 10 21:26:04.845331 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:04.845289    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:11.491015 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.490982    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:11.526488 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.526454    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"]
Sep 10 21:26:11.526729 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.526703    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc" podUID="1438f80e-78d0-4689-b67b-e6554c512453" containerName="payload-processing" containerID="cri-o://1cd4e37d61049946fd067bea647dca781801982e0870a40c7381e1e9188dc87e" gracePeriod=30
Sep 10 21:26:11.535809 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.535782    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"]
Sep 10 21:26:11.536470 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.536198    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd" podUID="98266168-b5f0-4542-8cce-f15eac2740e5" containerName="payload-pre-processing" containerID="cri-o://63b79e555c413c70847e873630b585c9699b65944c2293d2d0f7914457dfbf94" gracePeriod=30
Sep 10 21:26:11.576909 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.576884    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:11.636396 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.636370    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:11.703612 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.703561    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:11.766953 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.766880    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:11.810629 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.810606    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"
Sep 10 21:26:11.813699 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.813681    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"
Sep 10 21:26:11.957930 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.957898    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98266168-b5f0-4542-8cce-f15eac2740e5-plugins-config-volume\") pod \"98266168-b5f0-4542-8cce-f15eac2740e5\" (UID: \"98266168-b5f0-4542-8cce-f15eac2740e5\") "
Sep 10 21:26:11.958119 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.957936    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n545b\" (UniqueName: \"kubernetes.io/projected/1438f80e-78d0-4689-b67b-e6554c512453-kube-api-access-n545b\") pod \"1438f80e-78d0-4689-b67b-e6554c512453\" (UID: \"1438f80e-78d0-4689-b67b-e6554c512453\") "
Sep 10 21:26:11.958119 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.957982    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1438f80e-78d0-4689-b67b-e6554c512453-plugins-config-volume\") pod \"1438f80e-78d0-4689-b67b-e6554c512453\" (UID: \"1438f80e-78d0-4689-b67b-e6554c512453\") "
Sep 10 21:26:11.958119 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.958032    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdjdk\" (UniqueName: \"kubernetes.io/projected/98266168-b5f0-4542-8cce-f15eac2740e5-kube-api-access-zdjdk\") pod \"98266168-b5f0-4542-8cce-f15eac2740e5\" (UID: \"98266168-b5f0-4542-8cce-f15eac2740e5\") "
Sep 10 21:26:11.958302 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.958248    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98266168-b5f0-4542-8cce-f15eac2740e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "98266168-b5f0-4542-8cce-f15eac2740e5" (UID: "98266168-b5f0-4542-8cce-f15eac2740e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:11.958384 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.958368    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98266168-b5f0-4542-8cce-f15eac2740e5-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:26:11.958424 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.958387    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1438f80e-78d0-4689-b67b-e6554c512453-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1438f80e-78d0-4689-b67b-e6554c512453" (UID: "1438f80e-78d0-4689-b67b-e6554c512453"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:11.960016 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.959998    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98266168-b5f0-4542-8cce-f15eac2740e5-kube-api-access-zdjdk" (OuterVolumeSpecName: "kube-api-access-zdjdk") pod "98266168-b5f0-4542-8cce-f15eac2740e5" (UID: "98266168-b5f0-4542-8cce-f15eac2740e5"). InnerVolumeSpecName "kube-api-access-zdjdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:11.960088 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:11.960034    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1438f80e-78d0-4689-b67b-e6554c512453-kube-api-access-n545b" (OuterVolumeSpecName: "kube-api-access-n545b") pod "1438f80e-78d0-4689-b67b-e6554c512453" (UID: "1438f80e-78d0-4689-b67b-e6554c512453"). InnerVolumeSpecName "kube-api-access-n545b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:12.058773 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.058743    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdjdk\" (UniqueName: \"kubernetes.io/projected/98266168-b5f0-4542-8cce-f15eac2740e5-kube-api-access-zdjdk\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:26:12.058773 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.058768    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n545b\" (UniqueName: \"kubernetes.io/projected/1438f80e-78d0-4689-b67b-e6554c512453-kube-api-access-n545b\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:26:12.058969 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.058779    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1438f80e-78d0-4689-b67b-e6554c512453-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:26:12.089521 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.089489    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:12.512651 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.512571    2580 generic.go:356] "Generic (PLEG): container finished" podID="98266168-b5f0-4542-8cce-f15eac2740e5" containerID="63b79e555c413c70847e873630b585c9699b65944c2293d2d0f7914457dfbf94" exitCode=0
Sep 10 21:26:12.512651 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.512618    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"
Sep 10 21:26:12.513122 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.512658    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd" event={"ID":"98266168-b5f0-4542-8cce-f15eac2740e5","Type":"ContainerDied","Data":"63b79e555c413c70847e873630b585c9699b65944c2293d2d0f7914457dfbf94"}
Sep 10 21:26:12.513122 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.512712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd" event={"ID":"98266168-b5f0-4542-8cce-f15eac2740e5","Type":"ContainerDied","Data":"e252e8274000bf0def9010fa769f8a558d99416d8da1b4ec79f1738496baf9fb"}
Sep 10 21:26:12.513122 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.512735    2580 scope.go:117] "RemoveContainer" containerID="63b79e555c413c70847e873630b585c9699b65944c2293d2d0f7914457dfbf94"
Sep 10 21:26:12.514120 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.514093    2580 generic.go:356] "Generic (PLEG): container finished" podID="1438f80e-78d0-4689-b67b-e6554c512453" containerID="1cd4e37d61049946fd067bea647dca781801982e0870a40c7381e1e9188dc87e" exitCode=0
Sep 10 21:26:12.514227 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.514138    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc" event={"ID":"1438f80e-78d0-4689-b67b-e6554c512453","Type":"ContainerDied","Data":"1cd4e37d61049946fd067bea647dca781801982e0870a40c7381e1e9188dc87e"}
Sep 10 21:26:12.514227 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.514180    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc" event={"ID":"1438f80e-78d0-4689-b67b-e6554c512453","Type":"ContainerDied","Data":"b28417b1566b575437246317c88558fb0c2a896ccd51977613aa88b904abe60a"}
Sep 10 21:26:12.514227 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.514144    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"
Sep 10 21:26:12.523728 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.523714    2580 scope.go:117] "RemoveContainer" containerID="63b79e555c413c70847e873630b585c9699b65944c2293d2d0f7914457dfbf94"
Sep 10 21:26:12.523995 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:26:12.523972    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63b79e555c413c70847e873630b585c9699b65944c2293d2d0f7914457dfbf94\": container with ID starting with 63b79e555c413c70847e873630b585c9699b65944c2293d2d0f7914457dfbf94 not found: ID does not exist" containerID="63b79e555c413c70847e873630b585c9699b65944c2293d2d0f7914457dfbf94"
Sep 10 21:26:12.524052 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.524003    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b79e555c413c70847e873630b585c9699b65944c2293d2d0f7914457dfbf94"} err="failed to get container status \"63b79e555c413c70847e873630b585c9699b65944c2293d2d0f7914457dfbf94\": rpc error: code = NotFound desc = could not find container \"63b79e555c413c70847e873630b585c9699b65944c2293d2d0f7914457dfbf94\": container with ID starting with 63b79e555c413c70847e873630b585c9699b65944c2293d2d0f7914457dfbf94 not found: ID does not exist"
Sep 10 21:26:12.524052 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.524020    2580 scope.go:117] "RemoveContainer" containerID="1cd4e37d61049946fd067bea647dca781801982e0870a40c7381e1e9188dc87e"
Sep 10 21:26:12.532321 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.532179    2580 scope.go:117] "RemoveContainer" containerID="1cd4e37d61049946fd067bea647dca781801982e0870a40c7381e1e9188dc87e"
Sep 10 21:26:12.532456 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:26:12.532435    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cd4e37d61049946fd067bea647dca781801982e0870a40c7381e1e9188dc87e\": container with ID starting with 1cd4e37d61049946fd067bea647dca781801982e0870a40c7381e1e9188dc87e not found: ID does not exist" containerID="1cd4e37d61049946fd067bea647dca781801982e0870a40c7381e1e9188dc87e"
Sep 10 21:26:12.532499 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.532464    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cd4e37d61049946fd067bea647dca781801982e0870a40c7381e1e9188dc87e"} err="failed to get container status \"1cd4e37d61049946fd067bea647dca781801982e0870a40c7381e1e9188dc87e\": rpc error: code = NotFound desc = could not find container \"1cd4e37d61049946fd067bea647dca781801982e0870a40c7381e1e9188dc87e\": container with ID starting with 1cd4e37d61049946fd067bea647dca781801982e0870a40c7381e1e9188dc87e not found: ID does not exist"
Sep 10 21:26:12.536575 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.536554    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"]
Sep 10 21:26:12.540718 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.540698    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d24c8a-5b5ff8849d-mvzzd"]
Sep 10 21:26:12.550511 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.550491    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"]
Sep 10 21:26:12.554247 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.554227    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d24c8a-7bbc464b7f-kzxfc"]
Sep 10 21:26:12.957607 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.957577    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1438f80e-78d0-4689-b67b-e6554c512453" path="/var/lib/kubelet/pods/1438f80e-78d0-4689-b67b-e6554c512453/volumes"
Sep 10 21:26:12.957966 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:12.957952    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98266168-b5f0-4542-8cce-f15eac2740e5" path="/var/lib/kubelet/pods/98266168-b5f0-4542-8cce-f15eac2740e5/volumes"
Sep 10 21:26:14.650474 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:14.646104    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:26:18.300753 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:18.300719    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:18.357025 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:18.356996    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:18.650420 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:18.650342    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:22.669914 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:22.669882    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:35.647912 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:35.647874    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:26:37.961553 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:37.961514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:38.032764 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.032734    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"]
Sep 10 21:26:38.032975 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.032948    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9" podUID="ff076e71-30bc-4693-9853-2eb0acc8be6b" containerName="payload-pre-processing" containerID="cri-o://810e8515281f6ff6e50dd3c556c2d1e10d5697e9928c3bcf93719302c2e21b46" gracePeriod=30
Sep 10 21:26:38.053835 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.053810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:38.115295 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.115266    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:38.203440 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.203413    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:38.259640 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.259566    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:38.292363 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.292340    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"
Sep 10 21:26:38.385161 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.385132    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpzd2\" (UniqueName: \"kubernetes.io/projected/ff076e71-30bc-4693-9853-2eb0acc8be6b-kube-api-access-kpzd2\") pod \"ff076e71-30bc-4693-9853-2eb0acc8be6b\" (UID: \"ff076e71-30bc-4693-9853-2eb0acc8be6b\") "
Sep 10 21:26:38.385313 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.385258    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff076e71-30bc-4693-9853-2eb0acc8be6b-plugins-config-volume\") pod \"ff076e71-30bc-4693-9853-2eb0acc8be6b\" (UID: \"ff076e71-30bc-4693-9853-2eb0acc8be6b\") "
Sep 10 21:26:38.385671 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.385639    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff076e71-30bc-4693-9853-2eb0acc8be6b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ff076e71-30bc-4693-9853-2eb0acc8be6b" (UID: "ff076e71-30bc-4693-9853-2eb0acc8be6b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:38.387177 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.387151    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff076e71-30bc-4693-9853-2eb0acc8be6b-kube-api-access-kpzd2" (OuterVolumeSpecName: "kube-api-access-kpzd2") pod "ff076e71-30bc-4693-9853-2eb0acc8be6b" (UID: "ff076e71-30bc-4693-9853-2eb0acc8be6b"). InnerVolumeSpecName "kube-api-access-kpzd2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:38.486011 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.485983    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff076e71-30bc-4693-9853-2eb0acc8be6b-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:26:38.486011 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.486007    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpzd2\" (UniqueName: \"kubernetes.io/projected/ff076e71-30bc-4693-9853-2eb0acc8be6b-kube-api-access-kpzd2\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:26:38.606237 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.606207    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:38.652417 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.652390    2580 generic.go:356] "Generic (PLEG): container finished" podID="ff076e71-30bc-4693-9853-2eb0acc8be6b" containerID="810e8515281f6ff6e50dd3c556c2d1e10d5697e9928c3bcf93719302c2e21b46" exitCode=0
Sep 10 21:26:38.652526 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.652435    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"
Sep 10 21:26:38.652526 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.652473    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9" event={"ID":"ff076e71-30bc-4693-9853-2eb0acc8be6b","Type":"ContainerDied","Data":"810e8515281f6ff6e50dd3c556c2d1e10d5697e9928c3bcf93719302c2e21b46"}
Sep 10 21:26:38.652637 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.652525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9" event={"ID":"ff076e71-30bc-4693-9853-2eb0acc8be6b","Type":"ContainerDied","Data":"c099f475b055d0fc83d8ffd7cc528c042fff62619f2bec8e7f5287a34724a690"}
Sep 10 21:26:38.652637 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.652551    2580 scope.go:117] "RemoveContainer" containerID="810e8515281f6ff6e50dd3c556c2d1e10d5697e9928c3bcf93719302c2e21b46"
Sep 10 21:26:38.661740 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.661724    2580 scope.go:117] "RemoveContainer" containerID="810e8515281f6ff6e50dd3c556c2d1e10d5697e9928c3bcf93719302c2e21b46"
Sep 10 21:26:38.661996 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:26:38.661975    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"810e8515281f6ff6e50dd3c556c2d1e10d5697e9928c3bcf93719302c2e21b46\": container with ID starting with 810e8515281f6ff6e50dd3c556c2d1e10d5697e9928c3bcf93719302c2e21b46 not found: ID does not exist" containerID="810e8515281f6ff6e50dd3c556c2d1e10d5697e9928c3bcf93719302c2e21b46"
Sep 10 21:26:38.662078 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.662003    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810e8515281f6ff6e50dd3c556c2d1e10d5697e9928c3bcf93719302c2e21b46"} err="failed to get container status \"810e8515281f6ff6e50dd3c556c2d1e10d5697e9928c3bcf93719302c2e21b46\": rpc error: code = NotFound desc = could not find container \"810e8515281f6ff6e50dd3c556c2d1e10d5697e9928c3bcf93719302c2e21b46\": container with ID starting with 810e8515281f6ff6e50dd3c556c2d1e10d5697e9928c3bcf93719302c2e21b46 not found: ID does not exist"
Sep 10 21:26:38.674757 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.674735    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"]
Sep 10 21:26:38.678762 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.678741    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3bbf7d-7b9c797c4b-qtps9"]
Sep 10 21:26:38.956180 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:38.956118    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff076e71-30bc-4693-9853-2eb0acc8be6b" path="/var/lib/kubelet/pods/ff076e71-30bc-4693-9853-2eb0acc8be6b/volumes"
Sep 10 21:26:43.684246 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:43.684188    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:47.334076 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:26:47.334030    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:27:14.095309 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:14.095276    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:19.933649 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:19.933618    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:20.943197 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:20.943159    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:22.888038 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:22.887991    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:26.823402 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:26.823362    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:32.814280 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:32.814235    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:32.912072 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:32.912037    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:33.100272 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:33.100190    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:33.654625 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:33.654594    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:42.155536 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:42.155493    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:43.683643 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:43.683608    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:44.366452 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.366422    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"]
Sep 10 21:27:44.366667 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.366618    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb" podUID="6802adb0-289e-422e-b16f-4b2cc6a9a5b2" containerName="maas-api" containerID="cri-o://8177840284e5942357711e4f2b7f5a66639e4647c43dcc3c627a437f9e91de13" gracePeriod=30
Sep 10 21:27:44.384910 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.384874    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:44.427587 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.427554    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"]
Sep 10 21:27:44.427784 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.427765    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6" podUID="c6768016-bca0-45aa-a732-1d4f44e2fd79" containerName="payload-pre-processing" containerID="cri-o://707fc7c183c9360c5dc53d5041c445e305f9c41ba3df916653358d81b60e0bcc" gracePeriod=30
Sep 10 21:27:44.463314 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.463287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:44.536640 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.536613    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:44.638873 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.638840    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:44.688543 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.688513    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:27:44.691444 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.691424    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"
Sep 10 21:27:44.717190 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.717149    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:44.755255 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.755227    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-openshift-service-ca\") pod \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") "
Sep 10 21:27:44.755425 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.755324    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-metrics-certs\") pod \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") "
Sep 10 21:27:44.755425 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.755353    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-maas-api-tls\") pod \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") "
Sep 10 21:27:44.755425 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.755368    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g62gr\" (UniqueName: \"kubernetes.io/projected/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-kube-api-access-g62gr\") pod \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\" (UID: \"6802adb0-289e-422e-b16f-4b2cc6a9a5b2\") "
Sep 10 21:27:44.755657 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.755633    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6802adb0-289e-422e-b16f-4b2cc6a9a5b2" (UID: "6802adb0-289e-422e-b16f-4b2cc6a9a5b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:27:44.757472 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.757447    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-kube-api-access-g62gr" (OuterVolumeSpecName: "kube-api-access-g62gr") pod "6802adb0-289e-422e-b16f-4b2cc6a9a5b2" (UID: "6802adb0-289e-422e-b16f-4b2cc6a9a5b2"). InnerVolumeSpecName "kube-api-access-g62gr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:27:44.757571 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.757485    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6802adb0-289e-422e-b16f-4b2cc6a9a5b2" (UID: "6802adb0-289e-422e-b16f-4b2cc6a9a5b2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:27:44.757571 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.757497    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6802adb0-289e-422e-b16f-4b2cc6a9a5b2" (UID: "6802adb0-289e-422e-b16f-4b2cc6a9a5b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:27:44.856817 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.856786    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt6tr\" (UniqueName: \"kubernetes.io/projected/c6768016-bca0-45aa-a732-1d4f44e2fd79-kube-api-access-zt6tr\") pod \"c6768016-bca0-45aa-a732-1d4f44e2fd79\" (UID: \"c6768016-bca0-45aa-a732-1d4f44e2fd79\") "
Sep 10 21:27:44.856960 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.856944    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6768016-bca0-45aa-a732-1d4f44e2fd79-plugins-config-volume\") pod \"c6768016-bca0-45aa-a732-1d4f44e2fd79\" (UID: \"c6768016-bca0-45aa-a732-1d4f44e2fd79\") "
Sep 10 21:27:44.857260 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.857243    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-openshift-service-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:27:44.857309 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.857263    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-metrics-certs\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:27:44.857309 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.857273    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-maas-api-tls\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:27:44.857309 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.857281    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g62gr\" (UniqueName: \"kubernetes.io/projected/6802adb0-289e-422e-b16f-4b2cc6a9a5b2-kube-api-access-g62gr\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:27:44.857403 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.857323    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6768016-bca0-45aa-a732-1d4f44e2fd79-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c6768016-bca0-45aa-a732-1d4f44e2fd79" (UID: "c6768016-bca0-45aa-a732-1d4f44e2fd79"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:27:44.858705 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.858685    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6768016-bca0-45aa-a732-1d4f44e2fd79-kube-api-access-zt6tr" (OuterVolumeSpecName: "kube-api-access-zt6tr") pod "c6768016-bca0-45aa-a732-1d4f44e2fd79" (UID: "c6768016-bca0-45aa-a732-1d4f44e2fd79"). InnerVolumeSpecName "kube-api-access-zt6tr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:27:44.957980 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.957911    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6768016-bca0-45aa-a732-1d4f44e2fd79-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:27:44.957980 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.957936    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt6tr\" (UniqueName: \"kubernetes.io/projected/c6768016-bca0-45aa-a732-1d4f44e2fd79-kube-api-access-zt6tr\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:27:44.999527 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.999498    2580 generic.go:356] "Generic (PLEG): container finished" podID="c6768016-bca0-45aa-a732-1d4f44e2fd79" containerID="707fc7c183c9360c5dc53d5041c445e305f9c41ba3df916653358d81b60e0bcc" exitCode=0
Sep 10 21:27:44.999707 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.999536    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"
Sep 10 21:27:44.999707 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.999585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6" event={"ID":"c6768016-bca0-45aa-a732-1d4f44e2fd79","Type":"ContainerDied","Data":"707fc7c183c9360c5dc53d5041c445e305f9c41ba3df916653358d81b60e0bcc"}
Sep 10 21:27:44.999707 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.999648    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6" event={"ID":"c6768016-bca0-45aa-a732-1d4f44e2fd79","Type":"ContainerDied","Data":"c8c1d7adf810c25ee126d2f59b3ec8946e4adef61931de6a50cbd7b6c37c94d4"}
Sep 10 21:27:44.999707 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:44.999685    2580 scope.go:117] "RemoveContainer" containerID="707fc7c183c9360c5dc53d5041c445e305f9c41ba3df916653358d81b60e0bcc"
Sep 10 21:27:45.000926 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.000908    2580 generic.go:356] "Generic (PLEG): container finished" podID="6802adb0-289e-422e-b16f-4b2cc6a9a5b2" containerID="8177840284e5942357711e4f2b7f5a66639e4647c43dcc3c627a437f9e91de13" exitCode=0
Sep 10 21:27:45.001015 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.000995    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb" event={"ID":"6802adb0-289e-422e-b16f-4b2cc6a9a5b2","Type":"ContainerDied","Data":"8177840284e5942357711e4f2b7f5a66639e4647c43dcc3c627a437f9e91de13"}
Sep 10 21:27:45.001097 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.001024    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"
Sep 10 21:27:45.001097 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.001047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb" event={"ID":"6802adb0-289e-422e-b16f-4b2cc6a9a5b2","Type":"ContainerDied","Data":"172f4d59ee3b1122f5c41c3b57347a9dedc7e5811bca0763dfeafd7814bc9fc0"}
Sep 10 21:27:45.009401 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.009381    2580 scope.go:117] "RemoveContainer" containerID="707fc7c183c9360c5dc53d5041c445e305f9c41ba3df916653358d81b60e0bcc"
Sep 10 21:27:45.009689 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:27:45.009656    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"707fc7c183c9360c5dc53d5041c445e305f9c41ba3df916653358d81b60e0bcc\": container with ID starting with 707fc7c183c9360c5dc53d5041c445e305f9c41ba3df916653358d81b60e0bcc not found: ID does not exist" containerID="707fc7c183c9360c5dc53d5041c445e305f9c41ba3df916653358d81b60e0bcc"
Sep 10 21:27:45.009784 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.009696    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"707fc7c183c9360c5dc53d5041c445e305f9c41ba3df916653358d81b60e0bcc"} err="failed to get container status \"707fc7c183c9360c5dc53d5041c445e305f9c41ba3df916653358d81b60e0bcc\": rpc error: code = NotFound desc = could not find container \"707fc7c183c9360c5dc53d5041c445e305f9c41ba3df916653358d81b60e0bcc\": container with ID starting with 707fc7c183c9360c5dc53d5041c445e305f9c41ba3df916653358d81b60e0bcc not found: ID does not exist"
Sep 10 21:27:45.009784 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.009719    2580 scope.go:117] "RemoveContainer" containerID="8177840284e5942357711e4f2b7f5a66639e4647c43dcc3c627a437f9e91de13"
Sep 10 21:27:45.017786 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.017767    2580 scope.go:117] "RemoveContainer" containerID="8177840284e5942357711e4f2b7f5a66639e4647c43dcc3c627a437f9e91de13"
Sep 10 21:27:45.018251 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:27:45.018225    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8177840284e5942357711e4f2b7f5a66639e4647c43dcc3c627a437f9e91de13\": container with ID starting with 8177840284e5942357711e4f2b7f5a66639e4647c43dcc3c627a437f9e91de13 not found: ID does not exist" containerID="8177840284e5942357711e4f2b7f5a66639e4647c43dcc3c627a437f9e91de13"
Sep 10 21:27:45.018365 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.018281    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8177840284e5942357711e4f2b7f5a66639e4647c43dcc3c627a437f9e91de13"} err="failed to get container status \"8177840284e5942357711e4f2b7f5a66639e4647c43dcc3c627a437f9e91de13\": rpc error: code = NotFound desc = could not find container \"8177840284e5942357711e4f2b7f5a66639e4647c43dcc3c627a437f9e91de13\": container with ID starting with 8177840284e5942357711e4f2b7f5a66639e4647c43dcc3c627a437f9e91de13 not found: ID does not exist"
Sep 10 21:27:45.020702 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.020681    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"]
Sep 10 21:27:45.024657 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.024637    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a581b5-5db6d4dfd-9gldb"]
Sep 10 21:27:45.035164 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.035146    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"]
Sep 10 21:27:45.041565 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.041547    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a581b5-7dd6b98bb6-9xng6"]
Sep 10 21:27:45.056365 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:45.056332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:46.962717 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:46.962680    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6802adb0-289e-422e-b16f-4b2cc6a9a5b2" path="/var/lib/kubelet/pods/6802adb0-289e-422e-b16f-4b2cc6a9a5b2/volumes"
Sep 10 21:27:46.963154 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:46.963035    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6768016-bca0-45aa-a732-1d4f44e2fd79" path="/var/lib/kubelet/pods/c6768016-bca0-45aa-a732-1d4f44e2fd79/volumes"
Sep 10 21:27:47.829189 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:47.829157    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:27:50.994154 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:50.994121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:51.051763 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:51.051733    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:51.344284 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:51.344250    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:51.844566 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:27:51.844532    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:02.204857 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.204821    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"]
Sep 10 21:28:02.205665 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.205632    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff" podUID="2963a0f9-9413-4d70-b5d1-d4d79b8c37e5" containerName="maas-api" containerID="cri-o://6d4f380cd835a5493e2854a9df550f82177fb84856c914802d1dd01ec7414a65" gracePeriod=30
Sep 10 21:28:02.234050 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.234027    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:02.256411 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.256384    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"]
Sep 10 21:28:02.256633 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.256611    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9" podUID="8db3a326-fe52-47a1-8378-3bd8aebd7ae8" containerName="payload-processing" containerID="cri-o://a4f8b99c06ec0f347172e162de0129278dab20e48db2bb351d5523582ab958e1" gracePeriod=30
Sep 10 21:28:02.264180 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.264156    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"]
Sep 10 21:28:02.264354 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.264335    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp" podUID="8a14f6d0-eb22-4209-b663-3abf1479a686" containerName="payload-pre-processing" containerID="cri-o://0678cc9ba80e23c9c0605a2710ae37c34eef79e9c247c5c2affcfc6dd7402659" gracePeriod=30
Sep 10 21:28:02.301599 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.301563    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:02.364750 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.364716    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:02.426387 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.426214    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:02.484010 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.483973    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:02.538665 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.538639    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"
Sep 10 21:28:02.560364 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.560340    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:28:02.576987 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.576954    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"
Sep 10 21:28:02.708392 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.708364    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-metrics-certs\") pod \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") "
Sep 10 21:28:02.708584 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.708419    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-maas-api-tls\") pod \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") "
Sep 10 21:28:02.708584 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.708446    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a14f6d0-eb22-4209-b663-3abf1479a686-plugins-config-volume\") pod \"8a14f6d0-eb22-4209-b663-3abf1479a686\" (UID: \"8a14f6d0-eb22-4209-b663-3abf1479a686\") "
Sep 10 21:28:02.708584 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.708473    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db3a326-fe52-47a1-8378-3bd8aebd7ae8-plugins-config-volume\") pod \"8db3a326-fe52-47a1-8378-3bd8aebd7ae8\" (UID: \"8db3a326-fe52-47a1-8378-3bd8aebd7ae8\") "
Sep 10 21:28:02.708584 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.708515    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5xqb\" (UniqueName: \"kubernetes.io/projected/8a14f6d0-eb22-4209-b663-3abf1479a686-kube-api-access-b5xqb\") pod \"8a14f6d0-eb22-4209-b663-3abf1479a686\" (UID: \"8a14f6d0-eb22-4209-b663-3abf1479a686\") "
Sep 10 21:28:02.708584 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.708555    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pmr4\" (UniqueName: \"kubernetes.io/projected/8db3a326-fe52-47a1-8378-3bd8aebd7ae8-kube-api-access-5pmr4\") pod \"8db3a326-fe52-47a1-8378-3bd8aebd7ae8\" (UID: \"8db3a326-fe52-47a1-8378-3bd8aebd7ae8\") "
Sep 10 21:28:02.708845 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.708588    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-openshift-service-ca\") pod \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") "
Sep 10 21:28:02.708845 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.708619    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdhjq\" (UniqueName: \"kubernetes.io/projected/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-kube-api-access-xdhjq\") pod \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\" (UID: \"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5\") "
Sep 10 21:28:02.708942 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.708830    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a14f6d0-eb22-4209-b663-3abf1479a686-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a14f6d0-eb22-4209-b663-3abf1479a686" (UID: "8a14f6d0-eb22-4209-b663-3abf1479a686"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:02.708942 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.708839    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8db3a326-fe52-47a1-8378-3bd8aebd7ae8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8db3a326-fe52-47a1-8378-3bd8aebd7ae8" (UID: "8db3a326-fe52-47a1-8378-3bd8aebd7ae8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:02.709053 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.708975    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a14f6d0-eb22-4209-b663-3abf1479a686-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:28:02.709053 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.708993    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db3a326-fe52-47a1-8378-3bd8aebd7ae8-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:28:02.709295 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.709259    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2963a0f9-9413-4d70-b5d1-d4d79b8c37e5" (UID: "2963a0f9-9413-4d70-b5d1-d4d79b8c37e5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:02.711041 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.711015    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a14f6d0-eb22-4209-b663-3abf1479a686-kube-api-access-b5xqb" (OuterVolumeSpecName: "kube-api-access-b5xqb") pod "8a14f6d0-eb22-4209-b663-3abf1479a686" (UID: "8a14f6d0-eb22-4209-b663-3abf1479a686"). InnerVolumeSpecName "kube-api-access-b5xqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:02.711357 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.711338    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-kube-api-access-xdhjq" (OuterVolumeSpecName: "kube-api-access-xdhjq") pod "2963a0f9-9413-4d70-b5d1-d4d79b8c37e5" (UID: "2963a0f9-9413-4d70-b5d1-d4d79b8c37e5"). InnerVolumeSpecName "kube-api-access-xdhjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:02.711438 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.711339    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2963a0f9-9413-4d70-b5d1-d4d79b8c37e5" (UID: "2963a0f9-9413-4d70-b5d1-d4d79b8c37e5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:28:02.711438 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.711354    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db3a326-fe52-47a1-8378-3bd8aebd7ae8-kube-api-access-5pmr4" (OuterVolumeSpecName: "kube-api-access-5pmr4") pod "8db3a326-fe52-47a1-8378-3bd8aebd7ae8" (UID: "8db3a326-fe52-47a1-8378-3bd8aebd7ae8"). InnerVolumeSpecName "kube-api-access-5pmr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:02.711438 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.711353    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2963a0f9-9413-4d70-b5d1-d4d79b8c37e5" (UID: "2963a0f9-9413-4d70-b5d1-d4d79b8c37e5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:28:02.809646 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.809623    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-maas-api-tls\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:28:02.809646 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.809643    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5xqb\" (UniqueName: \"kubernetes.io/projected/8a14f6d0-eb22-4209-b663-3abf1479a686-kube-api-access-b5xqb\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:28:02.809778 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.809653    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pmr4\" (UniqueName: \"kubernetes.io/projected/8db3a326-fe52-47a1-8378-3bd8aebd7ae8-kube-api-access-5pmr4\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:28:02.809778 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.809661    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-openshift-service-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:28:02.809778 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.809671    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdhjq\" (UniqueName: \"kubernetes.io/projected/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-kube-api-access-xdhjq\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:28:02.809778 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.809679    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5-metrics-certs\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:28:02.821826 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:02.821806    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:03.094378 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.094310    2580 generic.go:356] "Generic (PLEG): container finished" podID="8db3a326-fe52-47a1-8378-3bd8aebd7ae8" containerID="a4f8b99c06ec0f347172e162de0129278dab20e48db2bb351d5523582ab958e1" exitCode=0
Sep 10 21:28:03.094378 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.094367    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"
Sep 10 21:28:03.094553 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.094391    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9" event={"ID":"8db3a326-fe52-47a1-8378-3bd8aebd7ae8","Type":"ContainerDied","Data":"a4f8b99c06ec0f347172e162de0129278dab20e48db2bb351d5523582ab958e1"}
Sep 10 21:28:03.094553 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.094444    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9" event={"ID":"8db3a326-fe52-47a1-8378-3bd8aebd7ae8","Type":"ContainerDied","Data":"85ed56c79cd99580df47a102d7b3e95feb28073bb8a7813a5e6b9f8c9041fdf7"}
Sep 10 21:28:03.094553 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.094475    2580 scope.go:117] "RemoveContainer" containerID="a4f8b99c06ec0f347172e162de0129278dab20e48db2bb351d5523582ab958e1"
Sep 10 21:28:03.095798 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.095721    2580 generic.go:356] "Generic (PLEG): container finished" podID="2963a0f9-9413-4d70-b5d1-d4d79b8c37e5" containerID="6d4f380cd835a5493e2854a9df550f82177fb84856c914802d1dd01ec7414a65" exitCode=0
Sep 10 21:28:03.095798 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.095776    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"
Sep 10 21:28:03.095932 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.095815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff" event={"ID":"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5","Type":"ContainerDied","Data":"6d4f380cd835a5493e2854a9df550f82177fb84856c914802d1dd01ec7414a65"}
Sep 10 21:28:03.095932 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.095865    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff" event={"ID":"2963a0f9-9413-4d70-b5d1-d4d79b8c37e5","Type":"ContainerDied","Data":"09a693ae0418e86b35ab06be5455c3ea7785f106f1c84a26cb3f297e11e56925"}
Sep 10 21:28:03.097630 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.097602    2580 generic.go:356] "Generic (PLEG): container finished" podID="8a14f6d0-eb22-4209-b663-3abf1479a686" containerID="0678cc9ba80e23c9c0605a2710ae37c34eef79e9c247c5c2affcfc6dd7402659" exitCode=0
Sep 10 21:28:03.097733 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.097650    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"
Sep 10 21:28:03.097733 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.097649    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp" event={"ID":"8a14f6d0-eb22-4209-b663-3abf1479a686","Type":"ContainerDied","Data":"0678cc9ba80e23c9c0605a2710ae37c34eef79e9c247c5c2affcfc6dd7402659"}
Sep 10 21:28:03.097830 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.097762    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp" event={"ID":"8a14f6d0-eb22-4209-b663-3abf1479a686","Type":"ContainerDied","Data":"a6558347a50129b3e99f7b4f739358ed3a09f587fb0a208680bff55a9fafbcab"}
Sep 10 21:28:03.105275 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.105257    2580 scope.go:117] "RemoveContainer" containerID="a4f8b99c06ec0f347172e162de0129278dab20e48db2bb351d5523582ab958e1"
Sep 10 21:28:03.105551 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:28:03.105533    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f8b99c06ec0f347172e162de0129278dab20e48db2bb351d5523582ab958e1\": container with ID starting with a4f8b99c06ec0f347172e162de0129278dab20e48db2bb351d5523582ab958e1 not found: ID does not exist" containerID="a4f8b99c06ec0f347172e162de0129278dab20e48db2bb351d5523582ab958e1"
Sep 10 21:28:03.105596 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.105557    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f8b99c06ec0f347172e162de0129278dab20e48db2bb351d5523582ab958e1"} err="failed to get container status \"a4f8b99c06ec0f347172e162de0129278dab20e48db2bb351d5523582ab958e1\": rpc error: code = NotFound desc = could not find container \"a4f8b99c06ec0f347172e162de0129278dab20e48db2bb351d5523582ab958e1\": container with ID starting with a4f8b99c06ec0f347172e162de0129278dab20e48db2bb351d5523582ab958e1 not found: ID does not exist"
Sep 10 21:28:03.105596 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.105573    2580 scope.go:117] "RemoveContainer" containerID="6d4f380cd835a5493e2854a9df550f82177fb84856c914802d1dd01ec7414a65"
Sep 10 21:28:03.113483 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.113466    2580 scope.go:117] "RemoveContainer" containerID="6d4f380cd835a5493e2854a9df550f82177fb84856c914802d1dd01ec7414a65"
Sep 10 21:28:03.113742 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:28:03.113726    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d4f380cd835a5493e2854a9df550f82177fb84856c914802d1dd01ec7414a65\": container with ID starting with 6d4f380cd835a5493e2854a9df550f82177fb84856c914802d1dd01ec7414a65 not found: ID does not exist" containerID="6d4f380cd835a5493e2854a9df550f82177fb84856c914802d1dd01ec7414a65"
Sep 10 21:28:03.113780 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.113749    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d4f380cd835a5493e2854a9df550f82177fb84856c914802d1dd01ec7414a65"} err="failed to get container status \"6d4f380cd835a5493e2854a9df550f82177fb84856c914802d1dd01ec7414a65\": rpc error: code = NotFound desc = could not find container \"6d4f380cd835a5493e2854a9df550f82177fb84856c914802d1dd01ec7414a65\": container with ID starting with 6d4f380cd835a5493e2854a9df550f82177fb84856c914802d1dd01ec7414a65 not found: ID does not exist"
Sep 10 21:28:03.113780 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.113766    2580 scope.go:117] "RemoveContainer" containerID="0678cc9ba80e23c9c0605a2710ae37c34eef79e9c247c5c2affcfc6dd7402659"
Sep 10 21:28:03.121318 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.121296    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"]
Sep 10 21:28:03.122349 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.122334    2580 scope.go:117] "RemoveContainer" containerID="0678cc9ba80e23c9c0605a2710ae37c34eef79e9c247c5c2affcfc6dd7402659"
Sep 10 21:28:03.122584 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:28:03.122567    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0678cc9ba80e23c9c0605a2710ae37c34eef79e9c247c5c2affcfc6dd7402659\": container with ID starting with 0678cc9ba80e23c9c0605a2710ae37c34eef79e9c247c5c2affcfc6dd7402659 not found: ID does not exist" containerID="0678cc9ba80e23c9c0605a2710ae37c34eef79e9c247c5c2affcfc6dd7402659"
Sep 10 21:28:03.122630 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.122589    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0678cc9ba80e23c9c0605a2710ae37c34eef79e9c247c5c2affcfc6dd7402659"} err="failed to get container status \"0678cc9ba80e23c9c0605a2710ae37c34eef79e9c247c5c2affcfc6dd7402659\": rpc error: code = NotFound desc = could not find container \"0678cc9ba80e23c9c0605a2710ae37c34eef79e9c247c5c2affcfc6dd7402659\": container with ID starting with 0678cc9ba80e23c9c0605a2710ae37c34eef79e9c247c5c2affcfc6dd7402659 not found: ID does not exist"
Sep 10 21:28:03.127534 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.127511    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0ac302-6899dc4db9-874n9"]
Sep 10 21:28:03.140106 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.140080    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"]
Sep 10 21:28:03.144187 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.144166    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0ac302-69578f6dc4-vsrrp"]
Sep 10 21:28:03.155999 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.155977    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"]
Sep 10 21:28:03.160040 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:03.160021    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-0ac302-774cbf444c-z6pff"]
Sep 10 21:28:04.956910 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:04.956838    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2963a0f9-9413-4d70-b5d1-d4d79b8c37e5" path="/var/lib/kubelet/pods/2963a0f9-9413-4d70-b5d1-d4d79b8c37e5/volumes"
Sep 10 21:28:04.957272 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:04.957212    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a14f6d0-eb22-4209-b663-3abf1479a686" path="/var/lib/kubelet/pods/8a14f6d0-eb22-4209-b663-3abf1479a686/volumes"
Sep 10 21:28:04.957528 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:04.957515    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8db3a326-fe52-47a1-8378-3bd8aebd7ae8" path="/var/lib/kubelet/pods/8db3a326-fe52-47a1-8378-3bd8aebd7ae8/volumes"
Sep 10 21:28:05.033632 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:05.033598    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:28:09.043406 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.043367    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:09.143016 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.142983    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:09.324903 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.324809    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:09.678841 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.678747    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc"]
Sep 10 21:28:09.679232 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679212    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a14f6d0-eb22-4209-b663-3abf1479a686" containerName="payload-pre-processing"
Sep 10 21:28:09.679320 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679234    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a14f6d0-eb22-4209-b663-3abf1479a686" containerName="payload-pre-processing"
Sep 10 21:28:09.679320 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679267    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6802adb0-289e-422e-b16f-4b2cc6a9a5b2" containerName="maas-api"
Sep 10 21:28:09.679320 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679276    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6802adb0-289e-422e-b16f-4b2cc6a9a5b2" containerName="maas-api"
Sep 10 21:28:09.679320 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679288    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db3a326-fe52-47a1-8378-3bd8aebd7ae8" containerName="payload-processing"
Sep 10 21:28:09.679320 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679297    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db3a326-fe52-47a1-8378-3bd8aebd7ae8" containerName="payload-processing"
Sep 10 21:28:09.679320 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679309    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2963a0f9-9413-4d70-b5d1-d4d79b8c37e5" containerName="maas-api"
Sep 10 21:28:09.679320 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679317    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2963a0f9-9413-4d70-b5d1-d4d79b8c37e5" containerName="maas-api"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679332    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1438f80e-78d0-4689-b67b-e6554c512453" containerName="payload-processing"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679341    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1438f80e-78d0-4689-b67b-e6554c512453" containerName="payload-processing"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679353    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff076e71-30bc-4693-9853-2eb0acc8be6b" containerName="payload-pre-processing"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679360    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff076e71-30bc-4693-9853-2eb0acc8be6b" containerName="payload-pre-processing"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679378    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6768016-bca0-45aa-a732-1d4f44e2fd79" containerName="payload-pre-processing"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679386    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6768016-bca0-45aa-a732-1d4f44e2fd79" containerName="payload-pre-processing"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679400    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98266168-b5f0-4542-8cce-f15eac2740e5" containerName="payload-pre-processing"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679408    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="98266168-b5f0-4542-8cce-f15eac2740e5" containerName="payload-pre-processing"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679502    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a14f6d0-eb22-4209-b663-3abf1479a686" containerName="payload-pre-processing"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679517    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1438f80e-78d0-4689-b67b-e6554c512453" containerName="payload-processing"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679529    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2963a0f9-9413-4d70-b5d1-d4d79b8c37e5" containerName="maas-api"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679540    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="98266168-b5f0-4542-8cce-f15eac2740e5" containerName="payload-pre-processing"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679551    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6768016-bca0-45aa-a732-1d4f44e2fd79" containerName="payload-pre-processing"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679564    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8db3a326-fe52-47a1-8378-3bd8aebd7ae8" containerName="payload-processing"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679575    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6802adb0-289e-422e-b16f-4b2cc6a9a5b2" containerName="maas-api"
Sep 10 21:28:09.679706 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.679583    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff076e71-30bc-4693-9853-2eb0acc8be6b" containerName="payload-pre-processing"
Sep 10 21:28:09.684517 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.684495    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc"
Sep 10 21:28:09.687177 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.687158    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:28:09.687299 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.687263    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p4gs5\""
Sep 10 21:28:09.690741 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.690716    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc"]
Sep 10 21:28:09.763043 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.763007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6342ece9-bac6-4989-927a-c236d5610d34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc\" (UID: \"6342ece9-bac6-4989-927a-c236d5610d34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc"
Sep 10 21:28:09.863479 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.863445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6342ece9-bac6-4989-927a-c236d5610d34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc\" (UID: \"6342ece9-bac6-4989-927a-c236d5610d34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc"
Sep 10 21:28:09.863940 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.863913    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:09.864195 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.864177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6342ece9-bac6-4989-927a-c236d5610d34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc\" (UID: \"6342ece9-bac6-4989-927a-c236d5610d34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc"
Sep 10 21:28:09.996787 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:09.996706    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc"
Sep 10 21:28:10.269321 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:10.269185    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc"]
Sep 10 21:28:11.146048 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:11.146006    2580 generic.go:356] "Generic (PLEG): container finished" podID="6342ece9-bac6-4989-927a-c236d5610d34" containerID="950eb47ae62e727e625011ea33cdcf535f08fa85735424d5ead3f75d99a07378" exitCode=0
Sep 10 21:28:11.146247 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:11.146092    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc" event={"ID":"6342ece9-bac6-4989-927a-c236d5610d34","Type":"ContainerDied","Data":"950eb47ae62e727e625011ea33cdcf535f08fa85735424d5ead3f75d99a07378"}
Sep 10 21:28:11.146247 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:11.146140    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc" event={"ID":"6342ece9-bac6-4989-927a-c236d5610d34","Type":"ContainerStarted","Data":"534da550313bde3142a87542a5a301b1e81ffa66118776d97e0f4ed01f9dcc7e"}
Sep 10 21:28:12.283325 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:12.283300    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc"
Sep 10 21:28:12.379778 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:12.379748    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6342ece9-bac6-4989-927a-c236d5610d34-maas-api-service-ca\") pod \"6342ece9-bac6-4989-927a-c236d5610d34\" (UID: \"6342ece9-bac6-4989-927a-c236d5610d34\") "
Sep 10 21:28:12.380093 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:12.380072    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6342ece9-bac6-4989-927a-c236d5610d34-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6342ece9-bac6-4989-927a-c236d5610d34" (UID: "6342ece9-bac6-4989-927a-c236d5610d34"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:12.480785 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:12.480705    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6342ece9-bac6-4989-927a-c236d5610d34-maas-api-service-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:28:13.158422 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:13.158394    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc"
Sep 10 21:28:13.158422 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:13.158408    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc" event={"ID":"6342ece9-bac6-4989-927a-c236d5610d34","Type":"ContainerDied","Data":"534da550313bde3142a87542a5a301b1e81ffa66118776d97e0f4ed01f9dcc7e"}
Sep 10 21:28:13.158633 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:13.158455    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="534da550313bde3142a87542a5a301b1e81ffa66118776d97e0f4ed01f9dcc7e"
Sep 10 21:28:20.276573 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.276492    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:20.314748 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.314718    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"]
Sep 10 21:28:20.314991 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.314960    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt" podUID="1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835" containerName="payload-pre-processing" containerID="cri-o://08c2a62851b65a350fd790d0010340ae288a2b260f80c330758371ee25088146" gracePeriod=30
Sep 10 21:28:20.339784 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.339758    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:20.394475 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.394442    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:20.462475 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.462443    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc"]
Sep 10 21:28:20.465637 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.465602    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:20.466701 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.466679    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-16bbec-r2xmc"]
Sep 10 21:28:20.521827 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.521790    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:20.580228 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.580208    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"
Sep 10 21:28:20.637743 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.637721    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hrml\" (UniqueName: \"kubernetes.io/projected/1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835-kube-api-access-9hrml\") pod \"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835\" (UID: \"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835\") "
Sep 10 21:28:20.637874 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.637799    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835-plugins-config-volume\") pod \"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835\" (UID: \"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835\") "
Sep 10 21:28:20.638194 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.638171    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835" (UID: "1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:28:20.639713 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.639696    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835-kube-api-access-9hrml" (OuterVolumeSpecName: "kube-api-access-9hrml") pod "1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835" (UID: "1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835"). InnerVolumeSpecName "kube-api-access-9hrml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:28:20.738581 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.738555    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hrml\" (UniqueName: \"kubernetes.io/projected/1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835-kube-api-access-9hrml\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:28:20.738581 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.738578    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835-plugins-config-volume\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:28:20.846337 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.846265    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:20.956552 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:20.956525    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6342ece9-bac6-4989-927a-c236d5610d34" path="/var/lib/kubelet/pods/6342ece9-bac6-4989-927a-c236d5610d34/volumes"
Sep 10 21:28:21.198493 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:21.198431    2580 generic.go:356] "Generic (PLEG): container finished" podID="1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835" containerID="08c2a62851b65a350fd790d0010340ae288a2b260f80c330758371ee25088146" exitCode=0
Sep 10 21:28:21.198493 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:21.198474    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"
Sep 10 21:28:21.198647 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:21.198516    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt" event={"ID":"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835","Type":"ContainerDied","Data":"08c2a62851b65a350fd790d0010340ae288a2b260f80c330758371ee25088146"}
Sep 10 21:28:21.198647 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:21.198571    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt" event={"ID":"1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835","Type":"ContainerDied","Data":"7e8a20ede7c35774f114b111910c30e51b859bb20d7c41241fb29c1c2dc979dc"}
Sep 10 21:28:21.198647 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:21.198600    2580 scope.go:117] "RemoveContainer" containerID="08c2a62851b65a350fd790d0010340ae288a2b260f80c330758371ee25088146"
Sep 10 21:28:21.208345 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:21.208327    2580 scope.go:117] "RemoveContainer" containerID="08c2a62851b65a350fd790d0010340ae288a2b260f80c330758371ee25088146"
Sep 10 21:28:21.208599 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:28:21.208581    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08c2a62851b65a350fd790d0010340ae288a2b260f80c330758371ee25088146\": container with ID starting with 08c2a62851b65a350fd790d0010340ae288a2b260f80c330758371ee25088146 not found: ID does not exist" containerID="08c2a62851b65a350fd790d0010340ae288a2b260f80c330758371ee25088146"
Sep 10 21:28:21.208652 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:21.208610    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08c2a62851b65a350fd790d0010340ae288a2b260f80c330758371ee25088146"} err="failed to get container status \"08c2a62851b65a350fd790d0010340ae288a2b260f80c330758371ee25088146\": rpc error: code = NotFound desc = could not find container \"08c2a62851b65a350fd790d0010340ae288a2b260f80c330758371ee25088146\": container with ID starting with 08c2a62851b65a350fd790d0010340ae288a2b260f80c330758371ee25088146 not found: ID does not exist"
Sep 10 21:28:21.214800 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:21.214776    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"]
Sep 10 21:28:21.217218 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:21.217195    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-16bbec-76d5476f4b-x94zt"]
Sep 10 21:28:22.939842 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:22.939803    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:28:22.956301 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:22.956274    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835" path="/var/lib/kubelet/pods/1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835/volumes"
Sep 10 21:28:26.858329 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:26.858289    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:26.911533 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:26.911495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:27.134851 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:27.134771    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:31.444795 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:28:31.444756    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:29:50.853151 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:29:50.853070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:52.136333 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:29:52.136297    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:29:53.119100 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:29:53.119069    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:54.704150 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:29:54.704120    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:30:21.037089 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:30:21.037040    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:30:28.937072 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:30:28.937014    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:30:37.292234 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:30:37.292196    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:30:55.437575 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:30:55.437539    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:31:03.732268 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:31:03.732232    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:31:37.384781 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:31:37.384710    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:37.473558 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:31:37.473526    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:42.625273 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:31:42.625229    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:27.832568 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:32:27.832539    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:32:36.029158 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:32:36.029125    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:32:44.059217 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:32:44.059178    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:46.548704 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:32:46.548668    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:33:07.482602 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:33:07.482566    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:10.334136 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:33:10.334100    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:33:32.223691 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:33:32.223649    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:32.275901 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:33:32.275871    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:41.314858 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:33:41.314827    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:42.836649 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:33:42.836621    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:43.434077 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:33:43.434024    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:33:52.133940 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:33:52.133903    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:33:59.130691 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:33:59.130654    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:34:05.926490 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:05.926455    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:34:06.155730 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.155697    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-554988677c-wrsvs"]
Sep 10 21:34:06.156094 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.156081    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6342ece9-bac6-4989-927a-c236d5610d34" containerName="revoke-keys"
Sep 10 21:34:06.156142 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.156096    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6342ece9-bac6-4989-927a-c236d5610d34" containerName="revoke-keys"
Sep 10 21:34:06.156142 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.156116    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835" containerName="payload-pre-processing"
Sep 10 21:34:06.156142 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.156121    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835" containerName="payload-pre-processing"
Sep 10 21:34:06.156268 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.156213    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f04b9d4-3ff6-4d1f-ac4d-fe52a2175835" containerName="payload-pre-processing"
Sep 10 21:34:06.156268 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.156222    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6342ece9-bac6-4989-927a-c236d5610d34" containerName="revoke-keys"
Sep 10 21:34:06.159460 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.159438    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.162912 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.162874    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 21:34:06.163017 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.162878    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-njbnl\""
Sep 10 21:34:06.163017 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.162936    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:34:06.163017 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.162915    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:34:06.163017 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.162881    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:34:06.171219 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.171196    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-554988677c-wrsvs"]
Sep 10 21:34:06.203256 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.203194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbb15808-e70d-4e83-b096-cf13d803657c-metrics-certs\") pod \"maas-api-554988677c-wrsvs\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") " pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.203256 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.203230    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbb15808-e70d-4e83-b096-cf13d803657c-maas-api-tls\") pod \"maas-api-554988677c-wrsvs\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") " pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.203398 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.203258    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzvtk\" (UniqueName: \"kubernetes.io/projected/bbb15808-e70d-4e83-b096-cf13d803657c-kube-api-access-jzvtk\") pod \"maas-api-554988677c-wrsvs\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") " pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.203398 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.203335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbb15808-e70d-4e83-b096-cf13d803657c-openshift-service-ca\") pod \"maas-api-554988677c-wrsvs\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") " pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.304127 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.304105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbb15808-e70d-4e83-b096-cf13d803657c-metrics-certs\") pod \"maas-api-554988677c-wrsvs\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") " pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.304245 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.304134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbb15808-e70d-4e83-b096-cf13d803657c-maas-api-tls\") pod \"maas-api-554988677c-wrsvs\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") " pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.304245 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.304160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzvtk\" (UniqueName: \"kubernetes.io/projected/bbb15808-e70d-4e83-b096-cf13d803657c-kube-api-access-jzvtk\") pod \"maas-api-554988677c-wrsvs\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") " pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.304245 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.304201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbb15808-e70d-4e83-b096-cf13d803657c-openshift-service-ca\") pod \"maas-api-554988677c-wrsvs\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") " pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.304876 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.304856    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbb15808-e70d-4e83-b096-cf13d803657c-openshift-service-ca\") pod \"maas-api-554988677c-wrsvs\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") " pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.306582 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.306557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbb15808-e70d-4e83-b096-cf13d803657c-metrics-certs\") pod \"maas-api-554988677c-wrsvs\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") " pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.306665 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.306583    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbb15808-e70d-4e83-b096-cf13d803657c-maas-api-tls\") pod \"maas-api-554988677c-wrsvs\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") " pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.311426 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.311408    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzvtk\" (UniqueName: \"kubernetes.io/projected/bbb15808-e70d-4e83-b096-cf13d803657c-kube-api-access-jzvtk\") pod \"maas-api-554988677c-wrsvs\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") " pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.471185 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.471102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:06.743938 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.743877    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-554988677c-wrsvs"]
Sep 10 21:34:06.746165 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:34:06.746126    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbb15808_e70d_4e83_b096_cf13d803657c.slice/crio-3a02bbaff60271e4ab83574cfe6f6c28570c83556482692b4f2e7c8dfbe785d6 WatchSource:0}: Error finding container 3a02bbaff60271e4ab83574cfe6f6c28570c83556482692b4f2e7c8dfbe785d6: Status 404 returned error can't find the container with id 3a02bbaff60271e4ab83574cfe6f6c28570c83556482692b4f2e7c8dfbe785d6
Sep 10 21:34:06.747579 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.747558    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:34:06.899891 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:06.899852    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs" event={"ID":"bbb15808-e70d-4e83-b096-cf13d803657c","Type":"ContainerStarted","Data":"3a02bbaff60271e4ab83574cfe6f6c28570c83556482692b4f2e7c8dfbe785d6"}
Sep 10 21:34:07.905859 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:07.905779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs" event={"ID":"bbb15808-e70d-4e83-b096-cf13d803657c","Type":"ContainerStarted","Data":"c26f0a224e31a7070f773b25d820a88b0a064f56d5ba8ef5649998d9ef1ad044"}
Sep 10 21:34:07.905859 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:07.905853    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:07.925860 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:07.925812    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs" podStartSLOduration=1.109312317 podStartE2EDuration="1.925796922s" totalImagesPullingTime="816.484605ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:34:07.925659679 +0000 UTC m=+2141.605158529" watchObservedRunningTime="2026-09-10 21:34:07.925796922 +0000 UTC m=+2141.605295771"
Sep 10 21:34:13.916076 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:13.916041    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:13.929418 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:13.929393    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:34:14.351618 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.351582    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"]
Sep 10 21:34:14.355535 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.355515    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.365625 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.365599    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"]
Sep 10 21:34:14.476744 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.476705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20171969-f014-4129-932b-d292d204767d-metrics-certs\") pod \"maas-api-65c67cbdfc-f9zlf\" (UID: \"20171969-f014-4129-932b-d292d204767d\") " pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.476915 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.476758    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20171969-f014-4129-932b-d292d204767d-openshift-service-ca\") pod \"maas-api-65c67cbdfc-f9zlf\" (UID: \"20171969-f014-4129-932b-d292d204767d\") " pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.476915 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.476786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20171969-f014-4129-932b-d292d204767d-maas-api-tls\") pod \"maas-api-65c67cbdfc-f9zlf\" (UID: \"20171969-f014-4129-932b-d292d204767d\") " pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.476915 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.476842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxsgt\" (UniqueName: \"kubernetes.io/projected/20171969-f014-4129-932b-d292d204767d-kube-api-access-nxsgt\") pod \"maas-api-65c67cbdfc-f9zlf\" (UID: \"20171969-f014-4129-932b-d292d204767d\") " pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.578260 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.578221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxsgt\" (UniqueName: \"kubernetes.io/projected/20171969-f014-4129-932b-d292d204767d-kube-api-access-nxsgt\") pod \"maas-api-65c67cbdfc-f9zlf\" (UID: \"20171969-f014-4129-932b-d292d204767d\") " pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.578419 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.578326    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20171969-f014-4129-932b-d292d204767d-metrics-certs\") pod \"maas-api-65c67cbdfc-f9zlf\" (UID: \"20171969-f014-4129-932b-d292d204767d\") " pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.578419 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.578356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20171969-f014-4129-932b-d292d204767d-openshift-service-ca\") pod \"maas-api-65c67cbdfc-f9zlf\" (UID: \"20171969-f014-4129-932b-d292d204767d\") " pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.578419 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.578371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20171969-f014-4129-932b-d292d204767d-maas-api-tls\") pod \"maas-api-65c67cbdfc-f9zlf\" (UID: \"20171969-f014-4129-932b-d292d204767d\") " pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.578984 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.578961    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20171969-f014-4129-932b-d292d204767d-openshift-service-ca\") pod \"maas-api-65c67cbdfc-f9zlf\" (UID: \"20171969-f014-4129-932b-d292d204767d\") " pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.580830 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.580809    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20171969-f014-4129-932b-d292d204767d-maas-api-tls\") pod \"maas-api-65c67cbdfc-f9zlf\" (UID: \"20171969-f014-4129-932b-d292d204767d\") " pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.580921 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.580860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20171969-f014-4129-932b-d292d204767d-metrics-certs\") pod \"maas-api-65c67cbdfc-f9zlf\" (UID: \"20171969-f014-4129-932b-d292d204767d\") " pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.585967 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.585938    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxsgt\" (UniqueName: \"kubernetes.io/projected/20171969-f014-4129-932b-d292d204767d-kube-api-access-nxsgt\") pod \"maas-api-65c67cbdfc-f9zlf\" (UID: \"20171969-f014-4129-932b-d292d204767d\") " pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.666846 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.666794    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:14.939912 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:14.939866    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"]
Sep 10 21:34:14.942327 ip-10-0-135-142 kubenswrapper[2580]: W0910 21:34:14.942293    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20171969_f014_4129_932b_d292d204767d.slice/crio-9553356fac219cad89ecf897683b596e37758ce02b5de5e35f738c13a1df13cc WatchSource:0}: Error finding container 9553356fac219cad89ecf897683b596e37758ce02b5de5e35f738c13a1df13cc: Status 404 returned error can't find the container with id 9553356fac219cad89ecf897683b596e37758ce02b5de5e35f738c13a1df13cc
Sep 10 21:34:15.948818 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:15.948780    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf" event={"ID":"20171969-f014-4129-932b-d292d204767d","Type":"ContainerStarted","Data":"0173648058d57c2de17b47377c25ed3c944f906d867af8ccfbfaa644e07c061d"}
Sep 10 21:34:15.949252 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:15.948831    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf" event={"ID":"20171969-f014-4129-932b-d292d204767d","Type":"ContainerStarted","Data":"9553356fac219cad89ecf897683b596e37758ce02b5de5e35f738c13a1df13cc"}
Sep 10 21:34:15.949252 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:15.948894    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:15.967334 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:15.967288    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf" podStartSLOduration=1.522483588 podStartE2EDuration="1.96727402s" totalImagesPullingTime="444.790432ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:34:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:34:15.965834189 +0000 UTC m=+2149.645333041" watchObservedRunningTime="2026-09-10 21:34:15.96727402 +0000 UTC m=+2149.646772870"
Sep 10 21:34:21.141641 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:21.141559    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:34:21.960281 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:21.960252    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-65c67cbdfc-f9zlf"
Sep 10 21:34:22.035442 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:22.035407    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-554988677c-wrsvs"]
Sep 10 21:34:22.035612 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:22.035589    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs" podUID="bbb15808-e70d-4e83-b096-cf13d803657c" containerName="maas-api" containerID="cri-o://c26f0a224e31a7070f773b25d820a88b0a064f56d5ba8ef5649998d9ef1ad044" gracePeriod=30
Sep 10 21:34:23.278078 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.278042    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:23.359121 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.359086    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbb15808-e70d-4e83-b096-cf13d803657c-maas-api-tls\") pod \"bbb15808-e70d-4e83-b096-cf13d803657c\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") "
Sep 10 21:34:23.359287 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.359140    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzvtk\" (UniqueName: \"kubernetes.io/projected/bbb15808-e70d-4e83-b096-cf13d803657c-kube-api-access-jzvtk\") pod \"bbb15808-e70d-4e83-b096-cf13d803657c\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") "
Sep 10 21:34:23.359287 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.359172    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbb15808-e70d-4e83-b096-cf13d803657c-metrics-certs\") pod \"bbb15808-e70d-4e83-b096-cf13d803657c\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") "
Sep 10 21:34:23.359287 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.359216    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbb15808-e70d-4e83-b096-cf13d803657c-openshift-service-ca\") pod \"bbb15808-e70d-4e83-b096-cf13d803657c\" (UID: \"bbb15808-e70d-4e83-b096-cf13d803657c\") "
Sep 10 21:34:23.359576 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.359550    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbb15808-e70d-4e83-b096-cf13d803657c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bbb15808-e70d-4e83-b096-cf13d803657c" (UID: "bbb15808-e70d-4e83-b096-cf13d803657c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:23.361407 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.361377    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbb15808-e70d-4e83-b096-cf13d803657c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bbb15808-e70d-4e83-b096-cf13d803657c" (UID: "bbb15808-e70d-4e83-b096-cf13d803657c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:34:23.361407 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.361397    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbb15808-e70d-4e83-b096-cf13d803657c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bbb15808-e70d-4e83-b096-cf13d803657c" (UID: "bbb15808-e70d-4e83-b096-cf13d803657c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:34:23.361530 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.361378    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbb15808-e70d-4e83-b096-cf13d803657c-kube-api-access-jzvtk" (OuterVolumeSpecName: "kube-api-access-jzvtk") pod "bbb15808-e70d-4e83-b096-cf13d803657c" (UID: "bbb15808-e70d-4e83-b096-cf13d803657c"). InnerVolumeSpecName "kube-api-access-jzvtk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:23.460237 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.460178    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbb15808-e70d-4e83-b096-cf13d803657c-maas-api-tls\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:34:23.460237 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.460201    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzvtk\" (UniqueName: \"kubernetes.io/projected/bbb15808-e70d-4e83-b096-cf13d803657c-kube-api-access-jzvtk\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:34:23.460237 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.460213    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbb15808-e70d-4e83-b096-cf13d803657c-metrics-certs\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:34:23.460237 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.460223    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbb15808-e70d-4e83-b096-cf13d803657c-openshift-service-ca\") on node \"ip-10-0-135-142.ec2.internal\" DevicePath \"\""
Sep 10 21:34:23.987733 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.987692    2580 generic.go:356] "Generic (PLEG): container finished" podID="bbb15808-e70d-4e83-b096-cf13d803657c" containerID="c26f0a224e31a7070f773b25d820a88b0a064f56d5ba8ef5649998d9ef1ad044" exitCode=0
Sep 10 21:34:23.987875 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.987738    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs"
Sep 10 21:34:23.987875 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.987739    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs" event={"ID":"bbb15808-e70d-4e83-b096-cf13d803657c","Type":"ContainerDied","Data":"c26f0a224e31a7070f773b25d820a88b0a064f56d5ba8ef5649998d9ef1ad044"}
Sep 10 21:34:23.987875 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.987860    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-554988677c-wrsvs" event={"ID":"bbb15808-e70d-4e83-b096-cf13d803657c","Type":"ContainerDied","Data":"3a02bbaff60271e4ab83574cfe6f6c28570c83556482692b4f2e7c8dfbe785d6"}
Sep 10 21:34:23.988003 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.987889    2580 scope.go:117] "RemoveContainer" containerID="c26f0a224e31a7070f773b25d820a88b0a064f56d5ba8ef5649998d9ef1ad044"
Sep 10 21:34:23.996830 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.996815    2580 scope.go:117] "RemoveContainer" containerID="c26f0a224e31a7070f773b25d820a88b0a064f56d5ba8ef5649998d9ef1ad044"
Sep 10 21:34:23.997190 ip-10-0-135-142 kubenswrapper[2580]: E0910 21:34:23.997171    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c26f0a224e31a7070f773b25d820a88b0a064f56d5ba8ef5649998d9ef1ad044\": container with ID starting with c26f0a224e31a7070f773b25d820a88b0a064f56d5ba8ef5649998d9ef1ad044 not found: ID does not exist" containerID="c26f0a224e31a7070f773b25d820a88b0a064f56d5ba8ef5649998d9ef1ad044"
Sep 10 21:34:23.997264 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:23.997199    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26f0a224e31a7070f773b25d820a88b0a064f56d5ba8ef5649998d9ef1ad044"} err="failed to get container status \"c26f0a224e31a7070f773b25d820a88b0a064f56d5ba8ef5649998d9ef1ad044\": rpc error: code = NotFound desc = could not find container \"c26f0a224e31a7070f773b25d820a88b0a064f56d5ba8ef5649998d9ef1ad044\": container with ID starting with c26f0a224e31a7070f773b25d820a88b0a064f56d5ba8ef5649998d9ef1ad044 not found: ID does not exist"
Sep 10 21:34:24.013395 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:24.013369    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-554988677c-wrsvs"]
Sep 10 21:34:24.016523 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:24.016501    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-554988677c-wrsvs"]
Sep 10 21:34:24.957507 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:24.957465    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbb15808-e70d-4e83-b096-cf13d803657c" path="/var/lib/kubelet/pods/bbb15808-e70d-4e83-b096-cf13d803657c/volumes"
Sep 10 21:34:26.831242 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:26.831207    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:34:27.336849 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:27.336819    2580 scope.go:117] "RemoveContainer" containerID="950eb47ae62e727e625011ea33cdcf535f08fa85735424d5ead3f75d99a07378"
Sep 10 21:34:33.836994 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:33.836964    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-mmj7x"]
Sep 10 21:34:41.922196 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:41.922161    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-chfbm_990f462f-3528-4812-823b-eb6b8fd4764a/manager/0.log"
Sep 10 21:34:42.040367 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:42.040334    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kvbb4_0da2211d-2f56-446f-991e-76bb767e7599/manager/0.log"
Sep 10 21:34:42.301730 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:42.301709    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fwhcj_355380ae-4412-4f96-a5f1-1e583dcfc2b1/server/0.log"
Sep 10 21:34:42.830314 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:42.830285    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d67fdbbcb-xt6xt_d11a2253-17f3-49f6-a19f-50b3be19957a/copy-manifests/0.log"
Sep 10 21:34:42.837630 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:42.837605    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d67fdbbcb-xt6xt_d11a2253-17f3-49f6-a19f-50b3be19957a/manager/0.log"
Sep 10 21:34:43.191992 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:43.191917    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-65c67cbdfc-f9zlf_20171969-f014-4129-932b-d292d204767d/maas-api/0.log"
Sep 10 21:34:43.423651 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:43.423626    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817915-thzld_4e906973-36db-499f-ae89-1137beed120e/cleanup/0.log"
Sep 10 21:34:44.594238 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:44.594203    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-xhb5c_60960b03-def6-4a77-8598-e098e49b5500/manager/0.log"
Sep 10 21:34:45.088589 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:45.088561    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-mmj7x_8b505246-b351-447c-aa7d-711bde6225e3/limitador/0.log"
Sep 10 21:34:45.706220 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:45.706189    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-lt9n9_3739b53b-8d76-48e0-9a2b-93b4d9d2445d/discovery/0.log"
Sep 10 21:34:45.945716 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:45.945685    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54c6c477f9-l8jqv_2560a72a-01b9-4cca-898a-cef77012090f/kube-auth-proxy/0.log"
Sep 10 21:34:46.196564 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:46.196534    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-q52r7_c795816d-ea80-46bf-9ed8-bace71464e54/payload-pre-processing/0.log"
Sep 10 21:34:47.044025 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:47.043995    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-mrm5b_a957e972-8424-4353-960a-1fc163fdfb2c/main/0.log"
Sep 10 21:34:47.406454 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:47.406380    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-54974_717b2af5-03e4-4fa2-9440-16be94a5c6fb/main/0.log"
Sep 10 21:34:53.305084 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:53.305025    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:54.699869 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:54.699835    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:56.204370 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:56.204336    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k7pdz_0819e2f5-b35c-4836-8d5e-939990ad6009/global-pull-secret-syncer/0.log"
Sep 10 21:34:56.321789 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:56.321762    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log"
Sep 10 21:34:56.427011 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:34:56.426987    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-142.ec2.internal_281550d9d955d980d569b392e95bf98a/haproxy/0.log"
Sep 10 21:35:00.422402 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:00.422372    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-xhb5c_60960b03-def6-4a77-8598-e098e49b5500/manager/0.log"
Sep 10 21:35:00.537921 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:00.537894    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-mmj7x_8b505246-b351-447c-aa7d-711bde6225e3/limitador/0.log"
Sep 10 21:35:02.409555 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:02.409520    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_310a9670-2b9b-4762-bcf4-bcc91747d22f/alertmanager/0.log"
Sep 10 21:35:02.433822 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:02.433798    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_310a9670-2b9b-4762-bcf4-bcc91747d22f/config-reloader/0.log"
Sep 10 21:35:02.453892 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:02.453874    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_310a9670-2b9b-4762-bcf4-bcc91747d22f/kube-rbac-proxy-web/0.log"
Sep 10 21:35:02.474090 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:02.474046    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_310a9670-2b9b-4762-bcf4-bcc91747d22f/kube-rbac-proxy/0.log"
Sep 10 21:35:02.496859 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:02.496837    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_310a9670-2b9b-4762-bcf4-bcc91747d22f/kube-rbac-proxy-metric/0.log"
Sep 10 21:35:02.522229 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:02.522205    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_310a9670-2b9b-4762-bcf4-bcc91747d22f/prom-label-proxy/0.log"
Sep 10 21:35:02.545645 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:02.545628    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_310a9670-2b9b-4762-bcf4-bcc91747d22f/init-config-reloader/0.log"
Sep 10 21:35:02.603944 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:02.603922    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-wj7qk_45829093-179f-41e5-b924-0046354ad7b0/kube-state-metrics/0.log"
Sep 10 21:35:02.625193 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:02.625175    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-wj7qk_45829093-179f-41e5-b924-0046354ad7b0/kube-rbac-proxy-main/0.log"
Sep 10 21:35:02.649960 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:02.649941    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-wj7qk_45829093-179f-41e5-b924-0046354ad7b0/kube-rbac-proxy-self/0.log"
Sep 10 21:35:02.897518 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:02.897500    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xs425_7cdc81b7-3624-4c4d-a96a-de94386c5a93/node-exporter/0.log"
Sep 10 21:35:02.916648 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:02.916630    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xs425_7cdc81b7-3624-4c4d-a96a-de94386c5a93/kube-rbac-proxy/0.log"
Sep 10 21:35:02.935871 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:02.935853    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xs425_7cdc81b7-3624-4c4d-a96a-de94386c5a93/init-textfile/0.log"
Sep 10 21:35:03.276719 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:03.276654    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-pft94_fd6c8a79-0fec-4e70-a85c-6613f42ad6b9/prometheus-operator-admission-webhook/0.log"
Sep 10 21:35:03.309270 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:03.309251    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f9d9f8445-7tj5p_99b5086c-47ec-4667-b239-0fd05ba7b65c/telemeter-client/0.log"
Sep 10 21:35:03.331605 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:03.331590    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f9d9f8445-7tj5p_99b5086c-47ec-4667-b239-0fd05ba7b65c/reload/0.log"
Sep 10 21:35:03.350952 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:03.350934    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f9d9f8445-7tj5p_99b5086c-47ec-4667-b239-0fd05ba7b65c/kube-rbac-proxy/0.log"
Sep 10 21:35:03.387731 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:03.387711    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5489f8965d-msdmg_51081d47-258b-4956-8961-c90f29034350/thanos-query/0.log"
Sep 10 21:35:03.406132 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:03.406116    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5489f8965d-msdmg_51081d47-258b-4956-8961-c90f29034350/kube-rbac-proxy-web/0.log"
Sep 10 21:35:03.434799 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:03.434783    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5489f8965d-msdmg_51081d47-258b-4956-8961-c90f29034350/kube-rbac-proxy/0.log"
Sep 10 21:35:03.453693 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:03.453676    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5489f8965d-msdmg_51081d47-258b-4956-8961-c90f29034350/prom-label-proxy/0.log"
Sep 10 21:35:03.475374 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:03.475358    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5489f8965d-msdmg_51081d47-258b-4956-8961-c90f29034350/kube-rbac-proxy-rules/0.log"
Sep 10 21:35:03.495123 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:03.495109    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5489f8965d-msdmg_51081d47-258b-4956-8961-c90f29034350/kube-rbac-proxy-metrics/0.log"
Sep 10 21:35:06.424289 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.424254    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:06.469301 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.469273    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-k9dkd_1d790ed2-e8f5-4eea-abd2-2af131a17bd8/download-server/0.log"
Sep 10 21:35:06.483132 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.483109    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ch2t_ba107ada-a3fd-4808-bfad-7105a2dc25f5/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:06.867841 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.867807    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"]
Sep 10 21:35:06.868274 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.868258    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbb15808-e70d-4e83-b096-cf13d803657c" containerName="maas-api"
Sep 10 21:35:06.868366 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.868277    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbb15808-e70d-4e83-b096-cf13d803657c" containerName="maas-api"
Sep 10 21:35:06.868429 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.868406    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbb15808-e70d-4e83-b096-cf13d803657c" containerName="maas-api"
Sep 10 21:35:06.871622 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.871604    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:06.874271 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.874252    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n84km\"/\"kube-root-ca.crt\""
Sep 10 21:35:06.875126 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.875110    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n84km\"/\"openshift-service-ca.crt\""
Sep 10 21:35:06.875218 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.875117    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-n84km\"/\"default-dockercfg-xm668\""
Sep 10 21:35:06.880009 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.879988    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"]
Sep 10 21:35:06.907950 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.907927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/262cae7e-4cf4-422e-ad69-8b3496784e27-lib-modules\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:06.908031 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.907958    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/262cae7e-4cf4-422e-ad69-8b3496784e27-podres\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:06.908031 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.907977    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcc27\" (UniqueName: \"kubernetes.io/projected/262cae7e-4cf4-422e-ad69-8b3496784e27-kube-api-access-pcc27\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:06.908134 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.908049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/262cae7e-4cf4-422e-ad69-8b3496784e27-proc\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:06.908134 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:06.908121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/262cae7e-4cf4-422e-ad69-8b3496784e27-sys\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:07.009418 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:07.009398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/262cae7e-4cf4-422e-ad69-8b3496784e27-podres\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:07.009516 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:07.009426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcc27\" (UniqueName: \"kubernetes.io/projected/262cae7e-4cf4-422e-ad69-8b3496784e27-kube-api-access-pcc27\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:07.009516 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:07.009456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/262cae7e-4cf4-422e-ad69-8b3496784e27-proc\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:07.009516 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:07.009500    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/262cae7e-4cf4-422e-ad69-8b3496784e27-sys\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:07.009629 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:07.009533    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/262cae7e-4cf4-422e-ad69-8b3496784e27-lib-modules\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:07.009629 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:07.009558    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/262cae7e-4cf4-422e-ad69-8b3496784e27-podres\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:07.009629 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:07.009607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/262cae7e-4cf4-422e-ad69-8b3496784e27-proc\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:07.009723 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:07.009636    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/262cae7e-4cf4-422e-ad69-8b3496784e27-sys\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:07.009723 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:07.009650    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/262cae7e-4cf4-422e-ad69-8b3496784e27-lib-modules\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:07.018159 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:07.018142    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcc27\" (UniqueName: \"kubernetes.io/projected/262cae7e-4cf4-422e-ad69-8b3496784e27-kube-api-access-pcc27\") pod \"perf-node-gather-daemonset-bjx64\" (UID: \"262cae7e-4cf4-422e-ad69-8b3496784e27\") " pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:07.183291 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:07.183236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:07.463123 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:07.461091    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"]
Sep 10 21:35:08.209883 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:08.209847    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64" event={"ID":"262cae7e-4cf4-422e-ad69-8b3496784e27","Type":"ContainerStarted","Data":"eba6211090b300fd58aed648a2e8c356ca2bd946b4d405ecab20b52c35f1f857"}
Sep 10 21:35:08.210039 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:08.209897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64" event={"ID":"262cae7e-4cf4-422e-ad69-8b3496784e27","Type":"ContainerStarted","Data":"712c99035a0897bc25fc879ee54083494186604fbd4d920033722585b7a1eb88"}
Sep 10 21:35:08.210039 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:08.209998    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:08.227689 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:08.227640    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64" podStartSLOduration=2.227627898 podStartE2EDuration="2.227627898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:35:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:35:08.226146444 +0000 UTC m=+2201.905645292" watchObservedRunningTime="2026-09-10 21:35:08.227627898 +0000 UTC m=+2201.907126804"
Sep 10 21:35:08.411248 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:08.411226    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h2hwr_c7d151d7-5a60-43b0-bb19-b89cdb84f169/dns/0.log"
Sep 10 21:35:08.431840 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:08.431818    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h2hwr_c7d151d7-5a60-43b0-bb19-b89cdb84f169/kube-rbac-proxy/0.log"
Sep 10 21:35:08.543387 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:08.543365    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xhglt_ed9cd22c-bd90-473a-9814-eea2735c3cf0/dns-node-resolver/0.log"
Sep 10 21:35:09.089673 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:09.089631    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4gz27_ea3e1564-e8e9-47db-a5e3-86002afeaeab/node-ca/0.log"
Sep 10 21:35:10.311732 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:10.311700    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-lt9n9_3739b53b-8d76-48e0-9a2b-93b4d9d2445d/discovery/0.log"
Sep 10 21:35:10.355292 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:10.355266    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54c6c477f9-l8jqv_2560a72a-01b9-4cca-898a-cef77012090f/kube-auth-proxy/0.log"
Sep 10 21:35:10.425887 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:10.425863    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-q52r7_c795816d-ea80-46bf-9ed8-bace71464e54/payload-pre-processing/0.log"
Sep 10 21:35:11.340473 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:11.340444    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6ntcm_8df9efce-a745-452e-a40d-60a5463e4537/serve-healthcheck-canary/0.log"
Sep 10 21:35:12.014218 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:12.014191    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vqzv4_b92e8363-eb18-4e60-8bce-9f5286beccfa/kube-rbac-proxy/0.log"
Sep 10 21:35:12.033444 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:12.033422    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vqzv4_b92e8363-eb18-4e60-8bce-9f5286beccfa/exporter/0.log"
Sep 10 21:35:12.052262 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:12.052244    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vqzv4_b92e8363-eb18-4e60-8bce-9f5286beccfa/extractor/0.log"
Sep 10 21:35:14.126029 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:14.126002    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-chfbm_990f462f-3528-4812-823b-eb6b8fd4764a/manager/0.log"
Sep 10 21:35:14.224457 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:14.224424    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-n84km/perf-node-gather-daemonset-bjx64"
Sep 10 21:35:14.290687 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:14.290658    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-kvbb4_0da2211d-2f56-446f-991e-76bb767e7599/manager/0.log"
Sep 10 21:35:14.364841 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:14.364819    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fwhcj_355380ae-4412-4f96-a5f1-1e583dcfc2b1/server/0.log"
Sep 10 21:35:14.598515 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:14.598488    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d67fdbbcb-xt6xt_d11a2253-17f3-49f6-a19f-50b3be19957a/manager/0.log"
Sep 10 21:35:14.618128 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:14.618105    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d67fdbbcb-xt6xt_d11a2253-17f3-49f6-a19f-50b3be19957a/copy-manifests/0.log"
Sep 10 21:35:16.029882 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:16.029805    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2cvzc_f4609476-6b4e-4675-8e83-2a135d5c5801/openshift-lws-operator/0.log"
Sep 10 21:35:16.747587 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:16.747514    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-lt9n9_3739b53b-8d76-48e0-9a2b-93b4d9d2445d/discovery/0.log"
Sep 10 21:35:16.775928 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:16.775906    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54c6c477f9-l8jqv_2560a72a-01b9-4cca-898a-cef77012090f/kube-auth-proxy/0.log"
Sep 10 21:35:16.827774 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:16.827751    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-q52r7_c795816d-ea80-46bf-9ed8-bace71464e54/payload-pre-processing/0.log"
Sep 10 21:35:18.549019 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:18.548989    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-lt9n9_3739b53b-8d76-48e0-9a2b-93b4d9d2445d/discovery/0.log"
Sep 10 21:35:18.575620 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:18.575551    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54c6c477f9-l8jqv_2560a72a-01b9-4cca-898a-cef77012090f/kube-auth-proxy/0.log"
Sep 10 21:35:18.632088 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:18.632052    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-q52r7_c795816d-ea80-46bf-9ed8-bace71464e54/payload-pre-processing/0.log"
Sep 10 21:35:22.551154 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:22.551117    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mxtq_059d85a3-7029-4c96-81b6-9a1cefe6d1ec/kube-multus-additional-cni-plugins/0.log"
Sep 10 21:35:22.572157 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:22.572130    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mxtq_059d85a3-7029-4c96-81b6-9a1cefe6d1ec/egress-router-binary-copy/0.log"
Sep 10 21:35:22.590973 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:22.590951    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mxtq_059d85a3-7029-4c96-81b6-9a1cefe6d1ec/cni-plugins/0.log"
Sep 10 21:35:22.609293 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:22.609271    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mxtq_059d85a3-7029-4c96-81b6-9a1cefe6d1ec/bond-cni-plugin/0.log"
Sep 10 21:35:22.626801 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:22.626780    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mxtq_059d85a3-7029-4c96-81b6-9a1cefe6d1ec/routeoverride-cni/0.log"
Sep 10 21:35:22.645439 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:22.645420    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mxtq_059d85a3-7029-4c96-81b6-9a1cefe6d1ec/whereabouts-cni-bincopy/0.log"
Sep 10 21:35:22.664508 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:22.664486    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8mxtq_059d85a3-7029-4c96-81b6-9a1cefe6d1ec/whereabouts-cni/0.log"
Sep 10 21:35:22.988565 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:22.988495    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-h9lm2_9837b724-5c2c-49df-b071-5e596f11562b/kube-multus/0.log"
Sep 10 21:35:23.131651 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:23.131620    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dkc7w_1bb0820e-193e-416f-a950-5eedb01fc196/network-metrics-daemon/0.log"
Sep 10 21:35:23.148642 ip-10-0-135-142 kubenswrapper[2580]: I0910 21:35:23.148621    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dkc7w_1bb0820e-193e-416f-a950-5eedb01fc196/kube-rbac-proxy/0.log"