Sep 17 03:19:31.540770 ip-10-0-137-141 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 03:19:31.943398 ip-10-0-137-141 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 03:19:31.943398 ip-10-0-137-141 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 03:19:31.943398 ip-10-0-137-141 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 03:19:31.943398 ip-10-0-137-141 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 03:19:31.943398 ip-10-0-137-141 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 03:19:31.945344 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.945246    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 03:19:31.950583 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950565    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 03:19:31.950583 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950581    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 03:19:31.950583 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950585    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 03:19:31.950583 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950588    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950591    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950595    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950597    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950600    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950603    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950605    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950608    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950615    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950618    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950622    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950625    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950628    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950632    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950636    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950639    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950642    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950644    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950647    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950650    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 03:19:31.950735 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950652    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950655    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950658    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950660    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950663    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950666    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950668    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950671    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950673    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950676    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950678    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950681    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950683    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950686    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950689    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950695    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950698    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950701    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950703    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 03:19:31.951216 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950706    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950709    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950733    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950738    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950742    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950745    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950748    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950751    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950754    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950757    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950760    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950763    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950766    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950769    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950771    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950774    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950777    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950779    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950782    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950784    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 03:19:31.951712 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950787    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950790    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950792    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950796    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950799    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950801    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950804    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950809    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950811    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950814    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950817    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950819    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950823    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950827    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950830    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950833    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950836    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950840    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950844    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 03:19:31.952207 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950846    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950849    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950851    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950854    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950857    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950859    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950861    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950864    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950867    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950869    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950873    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950875    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950878    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950880    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950883    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950885    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950890    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.950893    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951639    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951647    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 03:19:31.952701 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951651    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951654    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951658    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951662    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951665    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951668    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951671    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951674    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951677    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951679    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951682    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951685    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951688    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951690    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951693    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951696    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951699    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951701    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951704    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 03:19:31.953197 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951707    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951709    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951712    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951715    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951717    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951720    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951723    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951726    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951728    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951731    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951734    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951736    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951739    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951742    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951745    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951748    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951752    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951756    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951759    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 03:19:31.953685 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951763    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951766    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951769    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951772    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951774    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951777    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951780    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951782    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951785    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951788    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951790    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951792    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951795    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951798    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951800    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951803    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951806    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951808    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951811    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951814    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 03:19:31.954153 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951817    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951819    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951822    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951825    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951827    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951830    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951833    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951836    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951838    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951841    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951844    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951847    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951849    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951852    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951855    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951857    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951860    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951862    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951865    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951868    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 03:19:31.954660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951871    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951875    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951878    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951881    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951884    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951886    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951889    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951893    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951897    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951899    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951902    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951905    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951907    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951910    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951913    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951916    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951918    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951921    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.951923    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952319    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 03:19:31.955151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952324    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952327    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952330    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952333    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952337    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952341    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952344    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952347    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952350    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952353    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952355    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952358    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952361    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952363    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952366    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952368    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952371    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952373    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952376    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952379    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 03:19:31.955786 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952382    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952385    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952387    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952390    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952393    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952398    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952400    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952403    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952405    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952408    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952410    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952413    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952416    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952418    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952420    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952423    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952442    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952447    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952451    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 03:19:31.956275 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952455    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952459    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952463    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952466    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952468    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952471    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952473    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952476    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952479    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952481    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952484    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952486    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952489    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952492    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952494    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952497    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952500    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952502    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952508    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952511    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 03:19:31.956760 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952513    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952516    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952518    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952521    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952523    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952526    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952530    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952533    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952536    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952542    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952546    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952550    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952552    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952555    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952558    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952560    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952563    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952566    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952568    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 03:19:31.957300 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952571    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952573    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952576    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952578    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952582    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952584    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952587    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952590    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952592    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952595    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952598    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952602    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952605    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952607    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952610    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952612    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952615    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952618    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952621    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.952623    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 03:19:31.958021 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952695    2576 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952729    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952738    2576 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952743    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952748    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952752    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952756    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952760    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952763    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952766    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952770    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952773    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952776    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952779    2576 flags.go:64] FLAG: --cgroup-root=""
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952782    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952785    2576 flags.go:64] FLAG: --client-ca-file=""
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952788    2576 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952791    2576 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952795    2576 flags.go:64] FLAG: --cluster-domain=""
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952798    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952801    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952804    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952808    2576 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952815    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 03:19:31.958705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952818    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952821    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952825    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952828    2576 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952831    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952834    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952837    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952840    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952844    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952847    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952851    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952854    2576 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952857    2576 flags.go:64] FLAG: --enable-server="true"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952861    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952866    2576 flags.go:64] FLAG: --event-burst="100"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952869    2576 flags.go:64] FLAG: --event-qps="50"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952872    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952875    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952878    2576 flags.go:64] FLAG: --eviction-hard=""
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952881    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952884    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952887    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952890    2576 flags.go:64] FLAG: --eviction-soft=""
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952893    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952896    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 03:19:31.959327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952899    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952902    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952906    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952909    2576 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952912    2576 flags.go:64] FLAG: --feature-gates=""
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952916    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952919    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952922    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952925    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952928    2576 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952931    2576 flags.go:64] FLAG: --help="false"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952934    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-137-141.ec2.internal"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952937    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952940    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952943    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952946    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952949    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952953    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952956    2576 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952959    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952962    2576 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952966    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952969    2576 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952972    2576 flags.go:64] FLAG: --kube-reserved=""
Sep 17 03:19:31.959990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952975    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952978    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952981    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952984    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952986    2576 flags.go:64] FLAG: --lock-file=""
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952989    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952992    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.952995    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953000    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953003    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953006    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953009    2576 flags.go:64] FLAG: --logging-format="text"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953012    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953015    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953018    2576 flags.go:64] FLAG: --manifest-url=""
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953021    2576 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953025    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953028    2576 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953033    2576 flags.go:64] FLAG: --max-pods="110"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953036    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953038    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953041    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953044    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953047    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953050    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 03:19:31.960600 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953054    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953062    2576 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953065    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953068    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953071    2576 flags.go:64] FLAG: --pod-cidr=""
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953074    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953079    2576 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953082    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953085    2576 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953088    2576 flags.go:64] FLAG: --port="10250"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953091    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953094    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ed9be255cede6ab9"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953097    2576 flags.go:64] FLAG: --qos-reserved=""
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953100    2576 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953102    2576 flags.go:64] FLAG: --register-node="true"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953105    2576 flags.go:64] FLAG: --register-with-taints=""
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953109    2576 flags.go:64] FLAG: --registry-burst="10"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953112    2576 flags.go:64] FLAG: --registry-qps="5"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953115    2576 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953117    2576 flags.go:64] FLAG: --reserved-memory=""
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953121    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953124    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953127    2576 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953130    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953133    2576 flags.go:64] FLAG: --runonce="false"
Sep 17 03:19:31.961227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953136    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953139    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953142    2576 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953145    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953148    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953151    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953154    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953158    2576 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953161    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953164    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953167    2576 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953170    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953173    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953176    2576 flags.go:64] FLAG: --system-cgroups=""
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953179    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953185    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953187    2576 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953190    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953194    2576 flags.go:64] FLAG: --tls-min-version=""
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953197    2576 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953200    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953203    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953205    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953208    2576 flags.go:64] FLAG: --v="2"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953212    2576 flags.go:64] FLAG: --version="false"
Sep 17 03:19:31.961869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953218    2576 flags.go:64] FLAG: --vmodule=""
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953221    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953224    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953344    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953348    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953353    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953356    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953358    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953361    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953364    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953367    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953370    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953373    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953375    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953379    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953382    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953385    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953387    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953390    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953392    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 03:19:31.962526 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953395    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953399    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953401    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953404    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953407    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953410    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953413    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953415    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953418    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953421    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953423    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953438    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953443    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953446    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953448    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953451    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953454    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953456    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953459    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953462    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 03:19:31.963022 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953464    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953467    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953470    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953472    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953474    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953478    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953483    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953486    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953489    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953491    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953494    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953497    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953499    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953502    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953505    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953508    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953510    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953513    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953516    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953519    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 03:19:31.963552 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953521    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953524    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953526    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953529    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953533    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953536    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953538    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953541    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953543    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953546    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953549    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953551    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953554    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953557    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953559    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953562    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953564    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953567    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953571    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 03:19:31.964049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953575    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953579    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953582    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953585    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953588    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953590    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953593    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953596    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953599    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953602    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953606    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953610    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953613    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953616    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953619    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953622    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953625    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953629    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953631    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 03:19:31.964775 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953634    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953636    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953639    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.953642    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.953647    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.961154    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.961169    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.961186    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.961192    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961249    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961254    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961257    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961261    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961264    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 03:19:31.965489 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961267    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961270    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961272    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961275    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961278    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961281    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961284    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961287    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961290    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961292    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961295    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961298    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961300    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961303    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961306    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961308    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961310    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961314    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961317    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 03:19:31.965846 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961320    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961324    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961328    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961331    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961334    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961337    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961340    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961343    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961346    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961348    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961351    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961354    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961357    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961359    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961362    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961365    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961368    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961370    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961373    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 03:19:31.966382 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961376    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961378    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961382    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961385    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961387    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961390    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961393    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961395    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961398    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961401    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961404    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961407    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961410    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961414    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961417    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961420    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961423    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961440    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961445    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961449    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 03:19:31.966881 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961453    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961457    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961460    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961463    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961466    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961469    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961472    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961474    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961477    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961479    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961482    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961485    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961487    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961490    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961493    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961497    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961499    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961502    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961505    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961507    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 03:19:31.967376 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961510    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961514    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961518    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961522    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961525    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961528    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961530    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961533    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961535    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961538    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961540    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961543    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961546    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961549    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961551    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 03:19:31.967957 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961554    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.961559    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961674    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961678    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961681    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961684    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961687    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961691    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961695    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961698    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961700    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961703    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961706    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961709    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 03:19:31.968324 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961711    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961715    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961718    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961721    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961724    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961726    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961729    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961731    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961734    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961737    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961740    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961742    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961745    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961747    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961749    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961752    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961755    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961757    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961760    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 03:19:31.968690 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961762    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961765    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961767    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961769    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961772    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961774    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961776    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961779    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961782    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961784    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961787    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961789    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961792    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961796    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961798    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961801    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961803    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961806    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961808    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961811    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 03:19:31.969160 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961814    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961816    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961822    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961824    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961827    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961829    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961832    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961835    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961837    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961840    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961844    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961847    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961850    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961853    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961856    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961859    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961861    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961864    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961866    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 03:19:31.969665 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961869    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961872    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961875    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961878    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961881    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961883    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961886    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961889    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961892    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961895    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961898    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961901    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961903    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961906    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961909    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961913    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961915    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961918    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961920    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961923    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 03:19:31.970129 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961925    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961928    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961931    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961933    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961935    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961938    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961941    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961943    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:31.961946    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.961950    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.962063    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.963933    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.966387    2576 server.go:1019] "Starting client certificate rotation"
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.966467    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 03:19:31.970629 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.966506    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 03:19:31.970984 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.969906    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 03:19:31.971383 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.971361    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 03:19:31.981295 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.981272    2576 log.go:25] "Validated CRI v1 runtime API"
Sep 17 03:19:31.995867 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:31.995833    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 03:19:32.019572 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.019540    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qvm7c"
Sep 17 03:19:32.026543 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.026524    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qvm7c"
Sep 17 03:19:32.086487 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.086466    2576 log.go:25] "Validated CRI v1 image API"
Sep 17 03:19:32.087515 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.087495    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 03:19:32.090633 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.090611    2576 fs.go:135] Filesystem UUIDs: map[2edf0cf0-c21b-4728-802b-c7e3fc8330ff:/dev/nvme0n1p3 6950b4a7-ab5e-493c-a290-d1d15452c3f2:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 17 03:19:32.090687 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.090632    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 03:19:32.095653 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.095525    2576 manager.go:216] Machine: {Timestamp:2026-09-17 03:19:32.094320541 +0000 UTC m=+0.439046611 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100441 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23548fbebea5c0012d2a5d5d79340a SystemUUID:ec23548f-bebe-a5c0-012d-2a5d5d79340a BootID:1c18126e-f2df-4a9d-b0aa-c802ffedec85 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d9:08:41:57 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d9:08:41:57 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:f4:f1:9e:72:d8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 03:19:32.095653 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.095649    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 03:19:32.095792 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.095774    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 03:19:32.101380 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.101342    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 03:19:32.101584 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.101385    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-141.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 03:19:32.102311 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.102300    2576 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 03:19:32.102343 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.102314    2576 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 03:19:32.102343 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.102323    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 03:19:32.103071 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.103059    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 03:19:32.103891 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.103880    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 03:19:32.107727 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.107711    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 03:19:32.107864 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.107854    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 03:19:32.107904 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.107885    2576 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 03:19:32.107904 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.107897    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 03:19:32.107964 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.107912    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 03:19:32.107964 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.107923    2576 kubelet.go:398] "Adding apiserver pod source"
Sep 17 03:19:32.107964 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.107932    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 03:19:32.108337 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.108323    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 03:19:32.108386 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.108351    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 03:19:32.110593 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.110573    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 17 03:19:32.112988 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.112971    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 03:19:32.113063 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.113020    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 03:19:32.114554 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.114540    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 03:19:32.114597 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.114560    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 03:19:32.114597 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.114567    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 03:19:32.114597 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.114573    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 03:19:32.114597 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.114582    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 03:19:32.114597 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.114588    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 03:19:32.114733 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.114607    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 03:19:32.114733 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.114613    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 03:19:32.114733 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.114621    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 03:19:32.114733 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.114627    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 03:19:32.114733 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.114635    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 03:19:32.114733 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.114641    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 03:19:32.115296 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.115287    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 03:19:32.115325 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.115298    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 03:19:32.119684 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.119669    2576 server.go:1268] "Started kubelet"
Sep 17 03:19:32.119829 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.119795    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 03:19:32.119891 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.119773    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 03:19:32.120030 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.120015    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 03:19:32.120139 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.120123    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 03:19:32.120664 ip-10-0-137-141 systemd[1]: Started Kubernetes Kubelet.
Sep 17 03:19:32.121909 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.121877    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 03:19:32.122927 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.122901    2576 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 03:19:32.124330 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.124309    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 03:19:32.124418 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.124400    2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-141.ec2.internal" not found
Sep 17 03:19:32.124641 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:32.124619    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 03:19:32.126387 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.126369    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 03:19:32.126387 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.126379    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 03:19:32.126547 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:32.126472    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-141.ec2.internal\" not found"
Sep 17 03:19:32.127274 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.127257    2576 factory.go:55] Registering systemd factory
Sep 17 03:19:32.127274 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.127276    2576 factory.go:223] Registration of the systemd container factory successfully
Sep 17 03:19:32.127421 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.127400    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 03:19:32.127421 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.127404    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 03:19:32.127533 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.127449    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 03:19:32.127533 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.127487    2576 factory.go:153] Registering CRI-O factory
Sep 17 03:19:32.127533 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.127501    2576 factory.go:223] Registration of the crio container factory successfully
Sep 17 03:19:32.127668 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.127555    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 03:19:32.127668 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.127563    2576 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 03:19:32.127668 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.127569    2576 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 03:19:32.127668 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.127578    2576 factory.go:103] Registering Raw factory
Sep 17 03:19:32.127668 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.127592    2576 manager.go:1196] Started watching for new ooms in manager
Sep 17 03:19:32.128004 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.127990    2576 manager.go:349] Starting recovery of all containers
Sep 17 03:19:32.129751 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.129710    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 03:19:32.133070 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:32.133021    2576 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-141.ec2.internal\" not found" node="ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.139639 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.139490    2576 manager.go:354] Recovery completed
Sep 17 03:19:32.139743 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.139709    2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-141.ec2.internal" not found
Sep 17 03:19:32.145012 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.144998    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 03:19:32.146935 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.146919    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-141.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 03:19:32.147016 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.146948    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-141.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 03:19:32.147016 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.146959    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-141.ec2.internal" event="NodeHasSufficientPID"
Sep 17 03:19:32.147543 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.147530    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 03:19:32.147631 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.147543    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 03:19:32.147631 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.147555    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 03:19:32.147631 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.147563    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 03:19:32.148256 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.148239    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-141.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 03:19:32.148337 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.148273    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-141.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 03:19:32.148337 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.148289    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-141.ec2.internal" event="NodeHasSufficientPID"
Sep 17 03:19:32.149239 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.149225    2576 policy_none.go:49] "None policy: Start"
Sep 17 03:19:32.149307 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.149242    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 03:19:32.149307 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.149252    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 03:19:32.150200 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.150189    2576 policy_none.go:47] "Start"
Sep 17 03:19:32.189122 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.189107    2576 manager.go:337] "Starting Device Plugin manager"
Sep 17 03:19:32.242146 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:32.189145    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 03:19:32.242146 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.189155    2576 server.go:87] "Starting device plugin registration server"
Sep 17 03:19:32.242146 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.189545    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 03:19:32.242146 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.189556    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 03:19:32.242146 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.189656    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 03:19:32.242146 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.189757    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 03:19:32.242146 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.189766    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 03:19:32.242146 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:32.190406    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 03:19:32.242146 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:32.190510    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-141.ec2.internal\" not found"
Sep 17 03:19:32.242146 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.196199    2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-141.ec2.internal" not found
Sep 17 03:19:32.286317 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.286282    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 03:19:32.287543 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.287516    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 03:19:32.287664 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.287570    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 03:19:32.287664 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.287596    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 03:19:32.287763 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:32.287704    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 03:19:32.290079 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.290058    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 03:19:32.290205 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.290143    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 03:19:32.290862 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.290845    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-141.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 03:19:32.290955 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.290877    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-141.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 03:19:32.290955 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.290888    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-141.ec2.internal" event="NodeHasSufficientPID"
Sep 17 03:19:32.290955 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.290911    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.300051 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.300022    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.300051 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:32.300049    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-141.ec2.internal\": node \"ip-10-0-137-141.ec2.internal\" not found"
Sep 17 03:19:32.388178 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.388139    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-141.ec2.internal"]
Sep 17 03:19:32.390389 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.390372    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.391466 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.391448    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.417282 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.417252    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.422177 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.422159    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.428961 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.428939    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 03:19:32.430066 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.430048    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8541a873cbf1158c669335dcb651cecd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal\" (UID: \"8541a873cbf1158c669335dcb651cecd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.430142 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.430079    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8541a873cbf1158c669335dcb651cecd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal\" (UID: \"8541a873cbf1158c669335dcb651cecd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.430142 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.430107    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c7703ee05308cc145211e33cf98cc0b-config\") pod \"kube-apiserver-proxy-ip-10-0-137-141.ec2.internal\" (UID: \"1c7703ee05308cc145211e33cf98cc0b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.433300 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.433025    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 03:19:32.530953 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.530850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8541a873cbf1158c669335dcb651cecd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal\" (UID: \"8541a873cbf1158c669335dcb651cecd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.530953 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.530895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8541a873cbf1158c669335dcb651cecd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal\" (UID: \"8541a873cbf1158c669335dcb651cecd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.530953 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.530922    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c7703ee05308cc145211e33cf98cc0b-config\") pod \"kube-apiserver-proxy-ip-10-0-137-141.ec2.internal\" (UID: \"1c7703ee05308cc145211e33cf98cc0b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.530953 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.530954    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8541a873cbf1158c669335dcb651cecd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal\" (UID: \"8541a873cbf1158c669335dcb651cecd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.530953 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.530959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c7703ee05308cc145211e33cf98cc0b-config\") pod \"kube-apiserver-proxy-ip-10-0-137-141.ec2.internal\" (UID: \"1c7703ee05308cc145211e33cf98cc0b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.531221 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.530957    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8541a873cbf1158c669335dcb651cecd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal\" (UID: \"8541a873cbf1158c669335dcb651cecd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.731507 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.731477    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.740375 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.740348    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-141.ec2.internal"
Sep 17 03:19:32.964932 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.964827    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 03:19:32.965620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.965026    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 03:19:32.965620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.965026    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 03:19:32.965620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:32.965048    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 03:19:33.089133 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.089036    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 03:14:32 +0000 UTC" deadline="2028-02-19 07:53:21.01882537 +0000 UTC"
Sep 17 03:19:33.089133 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.089123    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12484h33m47.929704896s"
Sep 17 03:19:33.108237 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.108200    2576 apiserver.go:52] "Watching apiserver"
Sep 17 03:19:33.113421 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.113404    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 03:19:33.113789 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.113764    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-b24v9","openshift-network-operator/iptables-alerter-zggrr","kube-system/konnectivity-agent-65r5r","kube-system/kube-apiserver-proxy-ip-10-0-137-141.ec2.internal","openshift-dns/node-resolver-f9mkc","openshift-image-registry/node-ca-sp5r2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal","openshift-multus/multus-additional-cni-plugins-vwpxz","openshift-network-diagnostics/network-check-target-nwswx","openshift-ovn-kubernetes/ovnkube-node-w969r","kube-system/global-pull-secret-syncer-9cp4t","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n","openshift-cluster-node-tuning-operator/tuned-xrgcx","openshift-multus/multus-zxpp2"]
Sep 17 03:19:33.115329 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.115308    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:33.115454 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.115383    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:33.116479 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.116455    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zggrr"
Sep 17 03:19:33.117782 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.117764    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-65r5r"
Sep 17 03:19:33.118488 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.118471    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 03:19:33.118586 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.118471    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 03:19:33.118586 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.118512    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 03:19:33.118742 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.118730    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6ncgd\""
Sep 17 03:19:33.119061 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.119046    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f9mkc"
Sep 17 03:19:33.119584 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.119565    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5vpjv\""
Sep 17 03:19:33.119584 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.119579    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 03:19:33.119827 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.119812    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 03:19:33.120229 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.120214    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sp5r2"
Sep 17 03:19:33.120984 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.120968    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-j56rg\""
Sep 17 03:19:33.121099 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.121085    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 03:19:33.121165 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.121150    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 03:19:33.121588 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.121575    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.121881 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.121865    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 03:19:33.121881 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.121873    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 03:19:33.122022 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.122010    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 03:19:33.122074 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.122060    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qz5vk\""
Sep 17 03:19:33.122741 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.122723    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:33.122833 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.122789    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:33.123257 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.123242    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 03:19:33.123455 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.123441    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ll4qf\""
Sep 17 03:19:33.123525 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.123514    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 03:19:33.124625 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.123898    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 03:19:33.124625 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.124039    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 03:19:33.124625 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.124288    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 03:19:33.124807 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.124742    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.126393 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.126353    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:33.126626 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.126572    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:33.126693 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.126655    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 03:19:33.126803 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.126783    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 03:19:33.126852 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.126801    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 03:19:33.126852 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.126800    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c8mtd\""
Sep 17 03:19:33.126938 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.126910    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 03:19:33.128040 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.128026    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.129161 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.129145    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.129230 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.129189    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 03:19:33.129286 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.129254    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 03:19:33.130080 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.130063    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 03:19:33.130505 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.130488    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.130917 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.130901    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 03:19:33.130977 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.130900    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 03:19:33.131013 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.130976    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 03:19:33.131194 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.131169    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 03:19:33.131194 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.131169    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 03:19:33.131311 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.131225    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7vs6m\""
Sep 17 03:19:33.131311 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.131250    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fsfpl\""
Sep 17 03:19:33.134031 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.133963    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-node-log\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.134031 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.133995    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.134031 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134021    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-env-overrides\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.134172 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134045    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74vkk\" (UniqueName: \"kubernetes.io/projected/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-kube-api-access-74vkk\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.134172 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134069    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-device-dir\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.134172 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134083    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-sys-fs\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.134172 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134107    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158-host\") pod \"node-ca-sp5r2\" (UID: \"7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158\") " pod="openshift-image-registry/node-ca-sp5r2"
Sep 17 03:19:33.134172 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134134    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xktpg\" (UniqueName: \"kubernetes.io/projected/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-kube-api-access-xktpg\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.134172 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134167    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-run-systemd\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.134392 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134197    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-var-lib-openvswitch\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.134392 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134231    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 03:19:33.134392 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134236    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-dbus\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:33.134392 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134266    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-socket-dir\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.134392 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134292    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlkvz\" (UniqueName: \"kubernetes.io/projected/7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158-kube-api-access-mlkvz\") pod \"node-ca-sp5r2\" (UID: \"7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158\") " pod="openshift-image-registry/node-ca-sp5r2"
Sep 17 03:19:33.134392 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134317    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-run-netns\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.134392 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134339    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3047fc59-36fa-4bb2-a508-7d1a9a5edf35-host-slash\") pod \"iptables-alerter-zggrr\" (UID: \"3047fc59-36fa-4bb2-a508-7d1a9a5edf35\") " pod="openshift-network-operator/iptables-alerter-zggrr"
Sep 17 03:19:33.134392 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134361    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cf8486c3-9f66-4dd8-b244-634629e430fe-agent-certs\") pod \"konnectivity-agent-65r5r\" (UID: \"cf8486c3-9f66-4dd8-b244-634629e430fe\") " pod="kube-system/konnectivity-agent-65r5r"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134409    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cf8486c3-9f66-4dd8-b244-634629e430fe-konnectivity-ca\") pod \"konnectivity-agent-65r5r\" (UID: \"cf8486c3-9f66-4dd8-b244-634629e430fe\") " pod="kube-system/konnectivity-agent-65r5r"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134455    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158-serviceca\") pod \"node-ca-sp5r2\" (UID: \"7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158\") " pod="openshift-image-registry/node-ca-sp5r2"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134480    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-br6zv\""
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134486    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-system-cni-dir\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134512    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-cni-binary-copy\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134537    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-slash\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134560    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-ovnkube-config\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134587    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134612    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-cnibin\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134636    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134661    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134690    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64f9j\" (UniqueName: \"kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j\") pod \"network-check-target-nwswx\" (UID: \"6e66fbb6-8a12-4388-9b2b-4f4d256673cc\") " pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134726    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-systemd-units\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-etc-openvswitch\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.134784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134773    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3047fc59-36fa-4bb2-a508-7d1a9a5edf35-iptables-alerter-script\") pod \"iptables-alerter-zggrr\" (UID: \"3047fc59-36fa-4bb2-a508-7d1a9a5edf35\") " pod="openshift-network-operator/iptables-alerter-zggrr"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-742xd\" (UniqueName: \"kubernetes.io/projected/3047fc59-36fa-4bb2-a508-7d1a9a5edf35-kube-api-access-742xd\") pod \"iptables-alerter-zggrr\" (UID: \"3047fc59-36fa-4bb2-a508-7d1a9a5edf35\") " pod="openshift-network-operator/iptables-alerter-zggrr"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134851    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134874    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs\") pod \"network-metrics-daemon-b24v9\" (UID: \"b21d9f86-427c-4e22-89dd-f30258785552\") " pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134905    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-run-openvswitch\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-cni-bin\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134956    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-registration-dir\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.134991    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05b2116c-7c2b-4424-8038-a5580cac87b2-hosts-file\") pod \"node-resolver-f9mkc\" (UID: \"05b2116c-7c2b-4424-8038-a5580cac87b2\") " pod="openshift-dns/node-resolver-f9mkc"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.135014    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05b2116c-7c2b-4424-8038-a5580cac87b2-tmp-dir\") pod \"node-resolver-f9mkc\" (UID: \"05b2116c-7c2b-4424-8038-a5580cac87b2\") " pod="openshift-dns/node-resolver-f9mkc"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.135037    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l97wj\" (UniqueName: \"kubernetes.io/projected/b21d9f86-427c-4e22-89dd-f30258785552-kube-api-access-l97wj\") pod \"network-metrics-daemon-b24v9\" (UID: \"b21d9f86-427c-4e22-89dd-f30258785552\") " pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.135072    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-run-ovn\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.135100    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-ovn-node-metrics-cert\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.135117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-ovnkube-script-lib\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.135131    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-kubelet-config\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.135161    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-kubelet\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.135413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.135184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-log-socket\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.135934 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.135208    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-run-ovn-kubernetes\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.135934 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.135230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-cni-netd\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.135934 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.135253    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:33.135934 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.135277    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvz6v\" (UniqueName: \"kubernetes.io/projected/f52849ee-79a4-41a0-a215-702f7f31c3a1-kube-api-access-lvz6v\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.135934 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.135305    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb7mj\" (UniqueName: \"kubernetes.io/projected/05b2116c-7c2b-4424-8038-a5580cac87b2-kube-api-access-fb7mj\") pod \"node-resolver-f9mkc\" (UID: \"05b2116c-7c2b-4424-8038-a5580cac87b2\") " pod="openshift-dns/node-resolver-f9mkc"
Sep 17 03:19:33.145327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.145307    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 03:19:33.186895 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.186860    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-h75sh"
Sep 17 03:19:33.195792 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.195767    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-h75sh"
Sep 17 03:19:33.228131 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.228108    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 03:19:33.235730 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235705    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-device-dir\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.235852 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158-host\") pod \"node-ca-sp5r2\" (UID: \"7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158\") " pod="openshift-image-registry/node-ca-sp5r2"
Sep 17 03:19:33.235852 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-run-systemd\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.235852 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-var-lib-openvswitch\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.235852 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235795    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-multus-cni-dir\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.235852 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-var-lib-kubelet\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.235852 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-socket-dir\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235855    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158-host\") pod \"node-ca-sp5r2\" (UID: \"7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158\") " pod="openshift-image-registry/node-ca-sp5r2"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235873    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-var-lib-openvswitch\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235866    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlkvz\" (UniqueName: \"kubernetes.io/projected/7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158-kube-api-access-mlkvz\") pod \"node-ca-sp5r2\" (UID: \"7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158\") " pod="openshift-image-registry/node-ca-sp5r2"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235928    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-sys\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235943    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-device-dir\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-multus-socket-dir-parent\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cf8486c3-9f66-4dd8-b244-634629e430fe-agent-certs\") pod \"konnectivity-agent-65r5r\" (UID: \"cf8486c3-9f66-4dd8-b244-634629e430fe\") " pod="kube-system/konnectivity-agent-65r5r"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.235997    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-socket-dir\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236032    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cf8486c3-9f66-4dd8-b244-634629e430fe-konnectivity-ca\") pod \"konnectivity-agent-65r5r\" (UID: \"cf8486c3-9f66-4dd8-b244-634629e430fe\") " pod="kube-system/konnectivity-agent-65r5r"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236059    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158-serviceca\") pod \"node-ca-sp5r2\" (UID: \"7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158\") " pod="openshift-image-registry/node-ca-sp5r2"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236082    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-slash\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-ovnkube-config\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236115    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-modprobe-d\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.236124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236131    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-ovnkube-script-lib\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236147    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-kubernetes\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236207    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236223    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-cnibin\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236238    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236255    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236271    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-run-openvswitch\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236286    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-lib-modules\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236314    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-var-lib-kubelet\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236340    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnfts\" (UniqueName: \"kubernetes.io/projected/256101de-4250-47f7-9c68-0cb5d31c44a8-kube-api-access-mnfts\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236365    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs\") pod \"network-metrics-daemon-b24v9\" (UID: \"b21d9f86-427c-4e22-89dd-f30258785552\") " pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236411    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-system-cni-dir\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236448    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-var-lib-cni-bin\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-multus-conf-dir\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236492    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-multus-daemon-config\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236507    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 03:19:33.236798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236663    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cf8486c3-9f66-4dd8-b244-634629e430fe-konnectivity-ca\") pod \"konnectivity-agent-65r5r\" (UID: \"cf8486c3-9f66-4dd8-b244-634629e430fe\") " pod="kube-system/konnectivity-agent-65r5r"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236764    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236806    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-cnibin\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236828    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158-serviceca\") pod \"node-ca-sp5r2\" (UID: \"7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158\") " pod="openshift-image-registry/node-ca-sp5r2"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236886    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-slash\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236360    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-run-systemd\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236942    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236942    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236949    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-run-openvswitch\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236963    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-ovnkube-config\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.237021    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.236516    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-etc-kubernetes\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.237110    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs podName:b21d9f86-427c-4e22-89dd-f30258785552 nodeName:}" failed. No retries permitted until 2026-09-17 03:19:33.73707091 +0000 UTC m=+2.081796981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs") pod "network-metrics-daemon-b24v9" (UID: "b21d9f86-427c-4e22-89dd-f30258785552") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05b2116c-7c2b-4424-8038-a5580cac87b2-tmp-dir\") pod \"node-resolver-f9mkc\" (UID: \"05b2116c-7c2b-4424-8038-a5580cac87b2\") " pod="openshift-dns/node-resolver-f9mkc"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237167    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l97wj\" (UniqueName: \"kubernetes.io/projected/b21d9f86-427c-4e22-89dd-f30258785552-kube-api-access-l97wj\") pod \"network-metrics-daemon-b24v9\" (UID: \"b21d9f86-427c-4e22-89dd-f30258785552\") " pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237176    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-ovnkube-script-lib\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237194    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-run-k8s-cni-cncf-io\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.237630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-log-socket\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237266    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-sysconfig\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237290    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-run-netns\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237311    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-log-socket\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-node-log\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237352    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237379    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74vkk\" (UniqueName: \"kubernetes.io/projected/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-kube-api-access-74vkk\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237391    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-node-log\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-sys-fs\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237410    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.237414    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237446    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xktpg\" (UniqueName: \"kubernetes.io/projected/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-kube-api-access-xktpg\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-dbus\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237484    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-sys-fs\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.237525    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret podName:2ed49a17-07c3-4f80-8a00-7b5f21fbb61c nodeName:}" failed. No retries permitted until 2026-09-17 03:19:33.737481865 +0000 UTC m=+2.082207926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret") pod "global-pull-secret-syncer-9cp4t" (UID: "2ed49a17-07c3-4f80-8a00-7b5f21fbb61c") : object "kube-system"/"original-pull-secret" not registered
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237530    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-dbus\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:33.238534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-sysctl-d\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237581    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-hostroot\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qbtc\" (UniqueName: \"kubernetes.io/projected/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-kube-api-access-4qbtc\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237620    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-run-netns\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237646    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-run-netns\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237667    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-run\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3047fc59-36fa-4bb2-a508-7d1a9a5edf35-host-slash\") pod \"iptables-alerter-zggrr\" (UID: \"3047fc59-36fa-4bb2-a508-7d1a9a5edf35\") " pod="openshift-network-operator/iptables-alerter-zggrr"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237729    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3047fc59-36fa-4bb2-a508-7d1a9a5edf35-host-slash\") pod \"iptables-alerter-zggrr\" (UID: \"3047fc59-36fa-4bb2-a508-7d1a9a5edf35\") " pod="openshift-network-operator/iptables-alerter-zggrr"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237763    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-system-cni-dir\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237793    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-cni-binary-copy\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237819    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-ovn-node-metrics-cert\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237841    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-system-cni-dir\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237844    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-kubelet-config\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237867    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237874    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-tuned\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237912    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05b2116c-7c2b-4424-8038-a5580cac87b2-tmp-dir\") pod \"node-resolver-f9mkc\" (UID: \"05b2116c-7c2b-4424-8038-a5580cac87b2\") " pod="openshift-dns/node-resolver-f9mkc"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237919    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/256101de-4250-47f7-9c68-0cb5d31c44a8-tmp\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.239203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64f9j\" (UniqueName: \"kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j\") pod \"network-check-target-nwswx\" (UID: \"6e66fbb6-8a12-4388-9b2b-4f4d256673cc\") " pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.237986    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-systemd-units\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238000    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-kubelet-config\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238010    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-etc-openvswitch\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238034    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-cni-bin\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-cni-binary-copy\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238085    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3047fc59-36fa-4bb2-a508-7d1a9a5edf35-iptables-alerter-script\") pod \"iptables-alerter-zggrr\" (UID: \"3047fc59-36fa-4bb2-a508-7d1a9a5edf35\") " pod="openshift-network-operator/iptables-alerter-zggrr"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-742xd\" (UniqueName: \"kubernetes.io/projected/3047fc59-36fa-4bb2-a508-7d1a9a5edf35-kube-api-access-742xd\") pod \"iptables-alerter-zggrr\" (UID: \"3047fc59-36fa-4bb2-a508-7d1a9a5edf35\") " pod="openshift-network-operator/iptables-alerter-zggrr"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238135    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238160    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-cni-binary-copy\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238160    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-cni-bin\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238161    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-host\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238233    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-cnibin\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-run-multus-certs\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-registration-dir\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05b2116c-7c2b-4424-8038-a5580cac87b2-hosts-file\") pod \"node-resolver-f9mkc\" (UID: \"05b2116c-7c2b-4424-8038-a5580cac87b2\") " pod="openshift-dns/node-resolver-f9mkc"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238381    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-run-ovn\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.239707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238402    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05b2116c-7c2b-4424-8038-a5580cac87b2-hosts-file\") pod \"node-resolver-f9mkc\" (UID: \"05b2116c-7c2b-4424-8038-a5580cac87b2\") " pod="openshift-dns/node-resolver-f9mkc"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-etc-openvswitch\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-var-lib-cni-multus\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238499    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f52849ee-79a4-41a0-a215-702f7f31c3a1-registration-dir\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-kubelet\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238540    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-run-ovn-kubernetes\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238562    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3047fc59-36fa-4bb2-a508-7d1a9a5edf35-iptables-alerter-script\") pod \"iptables-alerter-zggrr\" (UID: \"3047fc59-36fa-4bb2-a508-7d1a9a5edf35\") " pod="openshift-network-operator/iptables-alerter-zggrr"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238567    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-cni-netd\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238540    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-kubelet\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238597    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-sysctl-conf\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238611    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-run-ovn\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238627    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-systemd\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238613    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-run-ovn-kubernetes\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238643    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-host-cni-netd\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238647    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238656    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvz6v\" (UniqueName: \"kubernetes.io/projected/f52849ee-79a4-41a0-a215-702f7f31c3a1-kube-api-access-lvz6v\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb7mj\" (UniqueName: \"kubernetes.io/projected/05b2116c-7c2b-4424-8038-a5580cac87b2-kube-api-access-fb7mj\") pod \"node-resolver-f9mkc\" (UID: \"05b2116c-7c2b-4424-8038-a5580cac87b2\") " pod="openshift-dns/node-resolver-f9mkc"
Sep 17 03:19:33.240203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-env-overrides\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.240679 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.238885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-systemd-units\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.240679 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.239049    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-env-overrides\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.240679 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.240288    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cf8486c3-9f66-4dd8-b244-634629e430fe-agent-certs\") pod \"konnectivity-agent-65r5r\" (UID: \"cf8486c3-9f66-4dd8-b244-634629e430fe\") " pod="kube-system/konnectivity-agent-65r5r"
Sep 17 03:19:33.240995 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.240979    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-ovn-node-metrics-cert\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.250799 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.250776    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlkvz\" (UniqueName: \"kubernetes.io/projected/7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158-kube-api-access-mlkvz\") pod \"node-ca-sp5r2\" (UID: \"7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158\") " pod="openshift-image-registry/node-ca-sp5r2"
Sep 17 03:19:33.250982 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.250960    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l97wj\" (UniqueName: \"kubernetes.io/projected/b21d9f86-427c-4e22-89dd-f30258785552-kube-api-access-l97wj\") pod \"network-metrics-daemon-b24v9\" (UID: \"b21d9f86-427c-4e22-89dd-f30258785552\") " pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:33.251289 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.251268    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74vkk\" (UniqueName: \"kubernetes.io/projected/ba9064f2-792b-4c2b-b804-5bd64bd69fc5-kube-api-access-74vkk\") pod \"ovnkube-node-w969r\" (UID: \"ba9064f2-792b-4c2b-b804-5bd64bd69fc5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.256664 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.256635    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 03:19:33.256664 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.256665    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 03:19:33.256854 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.256679    2576 projected.go:196] Error preparing data for projected volume kube-api-access-64f9j for pod openshift-network-diagnostics/network-check-target-nwswx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:19:33.256854 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.256764    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j podName:6e66fbb6-8a12-4388-9b2b-4f4d256673cc nodeName:}" failed. No retries permitted until 2026-09-17 03:19:33.756735353 +0000 UTC m=+2.101461429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-64f9j" (UniqueName: "kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j") pod "network-check-target-nwswx" (UID: "6e66fbb6-8a12-4388-9b2b-4f4d256673cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:19:33.256854 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.256767    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvz6v\" (UniqueName: \"kubernetes.io/projected/f52849ee-79a4-41a0-a215-702f7f31c3a1-kube-api-access-lvz6v\") pod \"aws-ebs-csi-driver-node-8d79n\" (UID: \"f52849ee-79a4-41a0-a215-702f7f31c3a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.257023 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.256970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-742xd\" (UniqueName: \"kubernetes.io/projected/3047fc59-36fa-4bb2-a508-7d1a9a5edf35-kube-api-access-742xd\") pod \"iptables-alerter-zggrr\" (UID: \"3047fc59-36fa-4bb2-a508-7d1a9a5edf35\") " pod="openshift-network-operator/iptables-alerter-zggrr"
Sep 17 03:19:33.257999 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.257978    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xktpg\" (UniqueName: \"kubernetes.io/projected/19037775-9be2-4a9a-aa31-9f1d9fe8aa74-kube-api-access-xktpg\") pod \"multus-additional-cni-plugins-vwpxz\" (UID: \"19037775-9be2-4a9a-aa31-9f1d9fe8aa74\") " pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.258763 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.258746    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb7mj\" (UniqueName: \"kubernetes.io/projected/05b2116c-7c2b-4424-8038-a5580cac87b2-kube-api-access-fb7mj\") pod \"node-resolver-f9mkc\" (UID: \"05b2116c-7c2b-4424-8038-a5580cac87b2\") " pod="openshift-dns/node-resolver-f9mkc"
Sep 17 03:19:33.264990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.264973    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n"
Sep 17 03:19:33.339671 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339639    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-tuned\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.339671 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339673    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/256101de-4250-47f7-9c68-0cb5d31c44a8-tmp\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.339902 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339710    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-cni-binary-copy\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.339902 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339731    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-host\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.339902 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339751    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-cnibin\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.339902 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339770    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-run-multus-certs\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.339902 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339793    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-var-lib-cni-multus\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.339902 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339817    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-sysctl-conf\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.339902 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-systemd\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.339902 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339882    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-cnibin\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.339902 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339900    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-multus-cni-dir\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-var-lib-kubelet\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339941    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-var-lib-cni-multus\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-sys\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339946    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-run-multus-certs\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339976    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-multus-socket-dir-parent\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340020    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-modprobe-d\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340059    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-kubernetes\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340073    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-sysctl-conf\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340088    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-lib-modules\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340127    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-var-lib-kubelet\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.339882    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-host\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340156    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-var-lib-kubelet\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340184    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-sys\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnfts\" (UniqueName: \"kubernetes.io/projected/256101de-4250-47f7-9c68-0cb5d31c44a8-kube-api-access-mnfts\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340222    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-lib-modules\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340233    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-system-cni-dir\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.340315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340263    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-cni-binary-copy\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340326    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-system-cni-dir\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-systemd\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340338    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-multus-cni-dir\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340397    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-kubernetes\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340420    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-modprobe-d\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340461    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-var-lib-kubelet\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340465    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-multus-socket-dir-parent\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-var-lib-cni-bin\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340512    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-multus-conf-dir\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340537    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-multus-daemon-config\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340554    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-var-lib-cni-bin\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340561    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-etc-kubernetes\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340578    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-multus-conf-dir\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340591    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-run-k8s-cni-cncf-io\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340622    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-etc-kubernetes\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340652    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-sysconfig\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340688    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-run-k8s-cni-cncf-io\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340711    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-sysconfig\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.341719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340716    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-run-netns\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340756    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-sysctl-d\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.341719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340774    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-hostroot\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340755    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-host-run-netns\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340809    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-hostroot\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340845    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qbtc\" (UniqueName: \"kubernetes.io/projected/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-kube-api-access-4qbtc\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-sysctl-d\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.341719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340886    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-run\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.341719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340947    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-multus-daemon-config\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.341719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.340955    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/256101de-4250-47f7-9c68-0cb5d31c44a8-run\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.342383 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.342369    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/256101de-4250-47f7-9c68-0cb5d31c44a8-tmp\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.342423 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.342403    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/256101de-4250-47f7-9c68-0cb5d31c44a8-etc-tuned\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.353601 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.353573    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qbtc\" (UniqueName: \"kubernetes.io/projected/c7063c41-ff67-4ce2-9d63-8cab1d847ba0-kube-api-access-4qbtc\") pod \"multus-zxpp2\" (UID: \"c7063c41-ff67-4ce2-9d63-8cab1d847ba0\") " pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.354938 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.354919    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnfts\" (UniqueName: \"kubernetes.io/projected/256101de-4250-47f7-9c68-0cb5d31c44a8-kube-api-access-mnfts\") pod \"tuned-xrgcx\" (UID: \"256101de-4250-47f7-9c68-0cb5d31c44a8\") " pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.365483 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:33.365450    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8541a873cbf1158c669335dcb651cecd.slice/crio-d40ab19bad18c1659272d22453e9f208c2264b94acafb93aace986503ca2db67 WatchSource:0}: Error finding container d40ab19bad18c1659272d22453e9f208c2264b94acafb93aace986503ca2db67: Status 404 returned error can't find the container with id d40ab19bad18c1659272d22453e9f208c2264b94acafb93aace986503ca2db67
Sep 17 03:19:33.365750 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:33.365731    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c7703ee05308cc145211e33cf98cc0b.slice/crio-5103278a29306021a37e42aa269f337169892a2c237693bef5e026dfd17b55e9 WatchSource:0}: Error finding container 5103278a29306021a37e42aa269f337169892a2c237693bef5e026dfd17b55e9: Status 404 returned error can't find the container with id 5103278a29306021a37e42aa269f337169892a2c237693bef5e026dfd17b55e9
Sep 17 03:19:33.369343 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.369326    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 03:19:33.455376 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.455342    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zggrr"
Sep 17 03:19:33.463917 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:33.463891    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3047fc59_36fa_4bb2_a508_7d1a9a5edf35.slice/crio-c7e9837aa263e888ae85a62b380bb3074a850e521c0f9c16c85a2c4a7793f9b7 WatchSource:0}: Error finding container c7e9837aa263e888ae85a62b380bb3074a850e521c0f9c16c85a2c4a7793f9b7: Status 404 returned error can't find the container with id c7e9837aa263e888ae85a62b380bb3074a850e521c0f9c16c85a2c4a7793f9b7
Sep 17 03:19:33.479964 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.479905    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-65r5r"
Sep 17 03:19:33.487114 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:33.487091    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf8486c3_9f66_4dd8_b244_634629e430fe.slice/crio-27754639513fe21050590281810e91e750d0de0d67144e5a1920939daa06f68d WatchSource:0}: Error finding container 27754639513fe21050590281810e91e750d0de0d67144e5a1920939daa06f68d: Status 404 returned error can't find the container with id 27754639513fe21050590281810e91e750d0de0d67144e5a1920939daa06f68d
Sep 17 03:19:33.493035 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.493018    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f9mkc"
Sep 17 03:19:33.494518 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.494500    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sp5r2"
Sep 17 03:19:33.526375 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.526349    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vwpxz"
Sep 17 03:19:33.533575 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:33.533555    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19037775_9be2_4a9a_aa31_9f1d9fe8aa74.slice/crio-28de7e2adb3a49ff6884349c90ce495e217bb168a3980b224f1b1ebf44d07b4a WatchSource:0}: Error finding container 28de7e2adb3a49ff6884349c90ce495e217bb168a3980b224f1b1ebf44d07b4a: Status 404 returned error can't find the container with id 28de7e2adb3a49ff6884349c90ce495e217bb168a3980b224f1b1ebf44d07b4a
Sep 17 03:19:33.548049 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.548028    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:33.555251 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:33.555228    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba9064f2_792b_4c2b_b804_5bd64bd69fc5.slice/crio-ac9fb84eceab9d2268e4b45f5f5c33170341e4009fbead3beee98412699de43f WatchSource:0}: Error finding container ac9fb84eceab9d2268e4b45f5f5c33170341e4009fbead3beee98412699de43f: Status 404 returned error can't find the container with id ac9fb84eceab9d2268e4b45f5f5c33170341e4009fbead3beee98412699de43f
Sep 17 03:19:33.573274 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.573255    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xrgcx"
Sep 17 03:19:33.578102 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.578080    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zxpp2"
Sep 17 03:19:33.581073 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:33.581050    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod256101de_4250_47f7_9c68_0cb5d31c44a8.slice/crio-ff29619b8a662b306506e6f8b8c6927fb24470939ce919a30b6c377c0fa994bc WatchSource:0}: Error finding container ff29619b8a662b306506e6f8b8c6927fb24470939ce919a30b6c377c0fa994bc: Status 404 returned error can't find the container with id ff29619b8a662b306506e6f8b8c6927fb24470939ce919a30b6c377c0fa994bc
Sep 17 03:19:33.585725 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:33.585701    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7063c41_ff67_4ce2_9d63_8cab1d847ba0.slice/crio-68be1723b527c2d3817786cb5fdf84e2a5748e776c12817f3cd355ba9d07d7ec WatchSource:0}: Error finding container 68be1723b527c2d3817786cb5fdf84e2a5748e776c12817f3cd355ba9d07d7ec: Status 404 returned error can't find the container with id 68be1723b527c2d3817786cb5fdf84e2a5748e776c12817f3cd355ba9d07d7ec
Sep 17 03:19:33.646836 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:19:33.646795    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52849ee_79a4_41a0_a215_702f7f31c3a1.slice/crio-0d4b1db808420dc26cf35b70fe281dcc5f2304856a6a72bb3aceb46a61b21883 WatchSource:0}: Error finding container 0d4b1db808420dc26cf35b70fe281dcc5f2304856a6a72bb3aceb46a61b21883: Status 404 returned error can't find the container with id 0d4b1db808420dc26cf35b70fe281dcc5f2304856a6a72bb3aceb46a61b21883
Sep 17 03:19:33.744291 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.744200    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs\") pod \"network-metrics-daemon-b24v9\" (UID: \"b21d9f86-427c-4e22-89dd-f30258785552\") " pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:33.744291 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.744239    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:33.744530 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.744344    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 03:19:33.744530 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.744359    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:19:33.744530 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.744394    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret podName:2ed49a17-07c3-4f80-8a00-7b5f21fbb61c nodeName:}" failed. No retries permitted until 2026-09-17 03:19:34.744381743 +0000 UTC m=+3.089107800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret") pod "global-pull-secret-syncer-9cp4t" (UID: "2ed49a17-07c3-4f80-8a00-7b5f21fbb61c") : object "kube-system"/"original-pull-secret" not registered
Sep 17 03:19:33.744530 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.744418    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs podName:b21d9f86-427c-4e22-89dd-f30258785552 nodeName:}" failed. No retries permitted until 2026-09-17 03:19:34.74440058 +0000 UTC m=+3.089126654 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs") pod "network-metrics-daemon-b24v9" (UID: "b21d9f86-427c-4e22-89dd-f30258785552") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:19:33.845648 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:33.845612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64f9j\" (UniqueName: \"kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j\") pod \"network-check-target-nwswx\" (UID: \"6e66fbb6-8a12-4388-9b2b-4f4d256673cc\") " pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:33.845824 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.845802    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 03:19:33.845877 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.845835    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 03:19:33.845877 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.845849    2576 projected.go:196] Error preparing data for projected volume kube-api-access-64f9j for pod openshift-network-diagnostics/network-check-target-nwswx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:19:33.845961 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:33.845915    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j podName:6e66fbb6-8a12-4388-9b2b-4f4d256673cc nodeName:}" failed. No retries permitted until 2026-09-17 03:19:34.845895072 +0000 UTC m=+3.190621144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-64f9j" (UniqueName: "kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j") pod "network-check-target-nwswx" (UID: "6e66fbb6-8a12-4388-9b2b-4f4d256673cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:19:34.033930 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.033836    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 03:19:34.074249 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.074212    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 03:19:34.199021 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.198951    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 03:14:33 +0000 UTC" deadline="2028-03-18 16:29:07.210773407 +0000 UTC"
Sep 17 03:19:34.199021 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.199019    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13165h9m33.011759674s"
Sep 17 03:19:34.262164 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.262125    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 03:19:34.291952 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.291119    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:34.291952 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:34.291252    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:34.319231 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.319147    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n" event={"ID":"f52849ee-79a4-41a0-a215-702f7f31c3a1","Type":"ContainerStarted","Data":"0d4b1db808420dc26cf35b70fe281dcc5f2304856a6a72bb3aceb46a61b21883"}
Sep 17 03:19:34.341493 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.341451    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zxpp2" event={"ID":"c7063c41-ff67-4ce2-9d63-8cab1d847ba0","Type":"ContainerStarted","Data":"68be1723b527c2d3817786cb5fdf84e2a5748e776c12817f3cd355ba9d07d7ec"}
Sep 17 03:19:34.361262 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.361215    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w969r" event={"ID":"ba9064f2-792b-4c2b-b804-5bd64bd69fc5","Type":"ContainerStarted","Data":"ac9fb84eceab9d2268e4b45f5f5c33170341e4009fbead3beee98412699de43f"}
Sep 17 03:19:34.378799 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.378764    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f9mkc" event={"ID":"05b2116c-7c2b-4424-8038-a5580cac87b2","Type":"ContainerStarted","Data":"6f92a665236579e63bf96478f4e3d5e571ae9c5fffb1d72586dfde99306871fe"}
Sep 17 03:19:34.410772 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.410593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-65r5r" event={"ID":"cf8486c3-9f66-4dd8-b244-634629e430fe","Type":"ContainerStarted","Data":"27754639513fe21050590281810e91e750d0de0d67144e5a1920939daa06f68d"}
Sep 17 03:19:34.423714 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.423674    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zggrr" event={"ID":"3047fc59-36fa-4bb2-a508-7d1a9a5edf35","Type":"ContainerStarted","Data":"c7e9837aa263e888ae85a62b380bb3074a850e521c0f9c16c85a2c4a7793f9b7"}
Sep 17 03:19:34.441909 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.441866    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal" event={"ID":"8541a873cbf1158c669335dcb651cecd","Type":"ContainerStarted","Data":"d40ab19bad18c1659272d22453e9f208c2264b94acafb93aace986503ca2db67"}
Sep 17 03:19:34.449619 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.449576    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xrgcx" event={"ID":"256101de-4250-47f7-9c68-0cb5d31c44a8","Type":"ContainerStarted","Data":"ff29619b8a662b306506e6f8b8c6927fb24470939ce919a30b6c377c0fa994bc"}
Sep 17 03:19:34.461103 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.461060    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vwpxz" event={"ID":"19037775-9be2-4a9a-aa31-9f1d9fe8aa74","Type":"ContainerStarted","Data":"28de7e2adb3a49ff6884349c90ce495e217bb168a3980b224f1b1ebf44d07b4a"}
Sep 17 03:19:34.479434 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.479395    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sp5r2" event={"ID":"7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158","Type":"ContainerStarted","Data":"02cb5ea28ee0e35afa7f093d06df3326c4a7dd3f20499a24e6b0f4ec514a6513"}
Sep 17 03:19:34.502623 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.502580    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-141.ec2.internal" event={"ID":"1c7703ee05308cc145211e33cf98cc0b","Type":"ContainerStarted","Data":"5103278a29306021a37e42aa269f337169892a2c237693bef5e026dfd17b55e9"}
Sep 17 03:19:34.752928 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.752250    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs\") pod \"network-metrics-daemon-b24v9\" (UID: \"b21d9f86-427c-4e22-89dd-f30258785552\") " pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:34.752928 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.752307    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:34.752928 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:34.752457    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 03:19:34.752928 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:34.752523    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret podName:2ed49a17-07c3-4f80-8a00-7b5f21fbb61c nodeName:}" failed. No retries permitted until 2026-09-17 03:19:36.752504155 +0000 UTC m=+5.097230216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret") pod "global-pull-secret-syncer-9cp4t" (UID: "2ed49a17-07c3-4f80-8a00-7b5f21fbb61c") : object "kube-system"/"original-pull-secret" not registered
Sep 17 03:19:34.752928 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:34.752457    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:19:34.752928 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:34.752600    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs podName:b21d9f86-427c-4e22-89dd-f30258785552 nodeName:}" failed. No retries permitted until 2026-09-17 03:19:36.752578914 +0000 UTC m=+5.097304977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs") pod "network-metrics-daemon-b24v9" (UID: "b21d9f86-427c-4e22-89dd-f30258785552") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:19:34.854218 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:34.853533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64f9j\" (UniqueName: \"kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j\") pod \"network-check-target-nwswx\" (UID: \"6e66fbb6-8a12-4388-9b2b-4f4d256673cc\") " pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:34.854218 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:34.853752    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 03:19:34.854218 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:34.853777    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 03:19:34.854218 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:34.853790    2576 projected.go:196] Error preparing data for projected volume kube-api-access-64f9j for pod openshift-network-diagnostics/network-check-target-nwswx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:19:34.854218 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:34.853852    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j podName:6e66fbb6-8a12-4388-9b2b-4f4d256673cc nodeName:}" failed. No retries permitted until 2026-09-17 03:19:36.853832901 +0000 UTC m=+5.198558961 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-64f9j" (UniqueName: "kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j") pod "network-check-target-nwswx" (UID: "6e66fbb6-8a12-4388-9b2b-4f4d256673cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:19:35.200193 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:35.200104    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 03:14:33 +0000 UTC" deadline="2028-03-21 07:53:20.970372672 +0000 UTC"
Sep 17 03:19:35.200193 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:35.200137    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13228h33m45.770238401s"
Sep 17 03:19:35.289132 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:35.288302    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:35.289132 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:35.288477    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:35.289132 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:35.288967    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:35.289132 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:35.289079    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:36.290627 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:36.290589    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:36.291098 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:36.290727    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:36.771107 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:36.770307    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs\") pod \"network-metrics-daemon-b24v9\" (UID: \"b21d9f86-427c-4e22-89dd-f30258785552\") " pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:36.771107 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:36.770641    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:19:36.771107 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:36.770715    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs podName:b21d9f86-427c-4e22-89dd-f30258785552 nodeName:}" failed. No retries permitted until 2026-09-17 03:19:40.770692914 +0000 UTC m=+9.115418982 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs") pod "network-metrics-daemon-b24v9" (UID: "b21d9f86-427c-4e22-89dd-f30258785552") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:19:36.771107 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:36.770733    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 03:19:36.771107 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:36.770787    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret podName:2ed49a17-07c3-4f80-8a00-7b5f21fbb61c nodeName:}" failed. No retries permitted until 2026-09-17 03:19:40.770770947 +0000 UTC m=+9.115497010 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret") pod "global-pull-secret-syncer-9cp4t" (UID: "2ed49a17-07c3-4f80-8a00-7b5f21fbb61c") : object "kube-system"/"original-pull-secret" not registered
Sep 17 03:19:36.771107 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:36.770639    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:36.872962 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:36.872119    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64f9j\" (UniqueName: \"kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j\") pod \"network-check-target-nwswx\" (UID: \"6e66fbb6-8a12-4388-9b2b-4f4d256673cc\") " pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:36.872962 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:36.872362    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 03:19:36.872962 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:36.872386    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 03:19:36.872962 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:36.872400    2576 projected.go:196] Error preparing data for projected volume kube-api-access-64f9j for pod openshift-network-diagnostics/network-check-target-nwswx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:19:36.872962 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:36.872473    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j podName:6e66fbb6-8a12-4388-9b2b-4f4d256673cc nodeName:}" failed. No retries permitted until 2026-09-17 03:19:40.872452723 +0000 UTC m=+9.217178790 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-64f9j" (UniqueName: "kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j") pod "network-check-target-nwswx" (UID: "6e66fbb6-8a12-4388-9b2b-4f4d256673cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:19:37.288961 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:37.288416    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:37.288961 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:37.288564    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:37.288961 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:37.288791    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:37.288961 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:37.288921    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:38.288727 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:38.288695    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:38.289198 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:38.288824    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:39.288488 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:39.288412    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:39.288647 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:39.288546    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:39.288754 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:39.288728    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:39.289136 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:39.288827    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:40.288478 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:40.288381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:40.288677 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:40.288540    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:40.807908 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:40.807632    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs\") pod \"network-metrics-daemon-b24v9\" (UID: \"b21d9f86-427c-4e22-89dd-f30258785552\") " pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:40.807908 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:40.807688    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:40.807908 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:40.807793    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:19:40.807908 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:40.807884    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs podName:b21d9f86-427c-4e22-89dd-f30258785552 nodeName:}" failed. No retries permitted until 2026-09-17 03:19:48.80785978 +0000 UTC m=+17.152585848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs") pod "network-metrics-daemon-b24v9" (UID: "b21d9f86-427c-4e22-89dd-f30258785552") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:19:40.808592 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:40.807934    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 03:19:40.808592 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:40.807985    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret podName:2ed49a17-07c3-4f80-8a00-7b5f21fbb61c nodeName:}" failed. No retries permitted until 2026-09-17 03:19:48.807970268 +0000 UTC m=+17.152696330 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret") pod "global-pull-secret-syncer-9cp4t" (UID: "2ed49a17-07c3-4f80-8a00-7b5f21fbb61c") : object "kube-system"/"original-pull-secret" not registered
Sep 17 03:19:40.908365 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:40.908328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64f9j\" (UniqueName: \"kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j\") pod \"network-check-target-nwswx\" (UID: \"6e66fbb6-8a12-4388-9b2b-4f4d256673cc\") " pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:40.908537 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:40.908517    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 03:19:40.908597 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:40.908544    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 03:19:40.908597 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:40.908558    2576 projected.go:196] Error preparing data for projected volume kube-api-access-64f9j for pod openshift-network-diagnostics/network-check-target-nwswx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:19:40.908662 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:40.908623    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j podName:6e66fbb6-8a12-4388-9b2b-4f4d256673cc nodeName:}" failed. No retries permitted until 2026-09-17 03:19:48.908603411 +0000 UTC m=+17.253329468 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-64f9j" (UniqueName: "kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j") pod "network-check-target-nwswx" (UID: "6e66fbb6-8a12-4388-9b2b-4f4d256673cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:19:41.288381 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:41.287893    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:41.288381 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:41.287928    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:41.288381 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:41.288035    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:41.288381 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:41.288138    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:42.288594 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:42.288559    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:42.289167 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:42.288677    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:43.288525 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:43.288489    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:43.288698 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:43.288489    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:43.288698 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:43.288626    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:43.289045 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:43.288747    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:44.288944 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:44.288901    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:44.289419 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:44.289052    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:45.288354 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:45.288316    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:45.288567 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:45.288326    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:45.288567 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:45.288473    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:45.288689 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:45.288579    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:46.288275 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:46.288240    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:46.288891 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:46.288367    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:47.288032 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:47.287993    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:47.288032 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:47.288026    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:47.288251 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:47.288147    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:47.288301 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:47.288274    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:48.288029 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:48.287975    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:48.288193 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:48.288123    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:48.870843 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:48.870805    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs\") pod \"network-metrics-daemon-b24v9\" (UID: \"b21d9f86-427c-4e22-89dd-f30258785552\") " pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:48.871337 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:48.870861    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:48.871337 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:48.870935    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:19:48.871337 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:48.870963    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 03:19:48.871337 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:48.871009    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs podName:b21d9f86-427c-4e22-89dd-f30258785552 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:04.870988214 +0000 UTC m=+33.215714281 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs") pod "network-metrics-daemon-b24v9" (UID: "b21d9f86-427c-4e22-89dd-f30258785552") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:19:48.871337 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:48.871032    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret podName:2ed49a17-07c3-4f80-8a00-7b5f21fbb61c nodeName:}" failed. No retries permitted until 2026-09-17 03:20:04.871020583 +0000 UTC m=+33.215746647 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret") pod "global-pull-secret-syncer-9cp4t" (UID: "2ed49a17-07c3-4f80-8a00-7b5f21fbb61c") : object "kube-system"/"original-pull-secret" not registered
Sep 17 03:19:48.971194 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:48.971163    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64f9j\" (UniqueName: \"kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j\") pod \"network-check-target-nwswx\" (UID: \"6e66fbb6-8a12-4388-9b2b-4f4d256673cc\") " pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:48.971378 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:48.971335    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 03:19:48.971378 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:48.971361    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 03:19:48.971378 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:48.971376    2576 projected.go:196] Error preparing data for projected volume kube-api-access-64f9j for pod openshift-network-diagnostics/network-check-target-nwswx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:19:48.971549 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:48.971462    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j podName:6e66fbb6-8a12-4388-9b2b-4f4d256673cc nodeName:}" failed. No retries permitted until 2026-09-17 03:20:04.971443658 +0000 UTC m=+33.316169731 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-64f9j" (UniqueName: "kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j") pod "network-check-target-nwswx" (UID: "6e66fbb6-8a12-4388-9b2b-4f4d256673cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:19:49.288760 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:49.288671    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:49.288760 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:49.288715    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:49.288983 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:49.288816    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:49.288983 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:49.288964    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:50.288507 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:50.288473    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:50.289070 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:50.288612    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:51.288138 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:51.288092    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:51.288138 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:51.288121    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:51.288373 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:51.288223    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:51.288373 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:51.288358    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:52.288787 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:52.288751    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:52.289597 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:52.288861    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:52.556268 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:52.556006    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w969r" event={"ID":"ba9064f2-792b-4c2b-b804-5bd64bd69fc5","Type":"ContainerStarted","Data":"bd763ee11454f60c6315612c6e8af5c058d14ba7ac43821f294d54ec7ca54a17"}
Sep 17 03:19:52.556443 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:52.556338    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w969r" event={"ID":"ba9064f2-792b-4c2b-b804-5bd64bd69fc5","Type":"ContainerStarted","Data":"f03ee4ecbafe5f040238022d0116d0f79ef6909bbec415df1398d9450ae9d97d"}
Sep 17 03:19:52.557750 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:52.557668    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xrgcx" event={"ID":"256101de-4250-47f7-9c68-0cb5d31c44a8","Type":"ContainerStarted","Data":"6aecb5113c6e6cac4ce3b4f017a395f0ce342126f06f348f4dbe91dbbb3f47e8"}
Sep 17 03:19:52.559166 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:52.559138    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-141.ec2.internal" event={"ID":"1c7703ee05308cc145211e33cf98cc0b","Type":"ContainerStarted","Data":"1f932dbb3be9361af7e8ff0497e6aa74e3ea83ca0b8040dce38a22cf27d1e280"}
Sep 17 03:19:52.583898 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:52.583834    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xrgcx" podStartSLOduration=2.079211398 podStartE2EDuration="20.58381539s" totalImagesPullingTime="18.504603992s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:19:52.583640973 +0000 UTC m=+20.928367052" watchObservedRunningTime="2026-09-17 03:19:52.58381539 +0000 UTC m=+20.928541473"
Sep 17 03:19:52.594812 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:52.594766    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-141.ec2.internal" podStartSLOduration=20.594749327 podStartE2EDuration="20.594749327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:19:52.594586047 +0000 UTC m=+20.939312146" watchObservedRunningTime="2026-09-17 03:19:52.594749327 +0000 UTC m=+20.939475402"
Sep 17 03:19:53.288138 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.287954    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:53.288309 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.287992    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:53.288309 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:53.288233    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:53.288404 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:53.288313    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:53.562989 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.562952    2576 generic.go:356] "Generic (PLEG): container finished" podID="19037775-9be2-4a9a-aa31-9f1d9fe8aa74" containerID="3db7e1c00a852e67c25807dd67d7358b41028d77fbc3bc054bc79f7e1fdd6632" exitCode=0
Sep 17 03:19:53.563640 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.563038    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vwpxz" event={"ID":"19037775-9be2-4a9a-aa31-9f1d9fe8aa74","Type":"ContainerDied","Data":"3db7e1c00a852e67c25807dd67d7358b41028d77fbc3bc054bc79f7e1fdd6632"}
Sep 17 03:19:53.564783 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.564748    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sp5r2" event={"ID":"7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158","Type":"ContainerStarted","Data":"f858ab46bf5b84dc9aa650340e9f72022c4d3170e8c081c6591325c38ef22fee"}
Sep 17 03:19:53.566676 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.566645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n" event={"ID":"f52849ee-79a4-41a0-a215-702f7f31c3a1","Type":"ContainerStarted","Data":"46d984cb84047673cf57886c458bb1ea474c224a9b05bb9f8f6ec9e7191a01ab"}
Sep 17 03:19:53.568307 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.568280    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zxpp2" event={"ID":"c7063c41-ff67-4ce2-9d63-8cab1d847ba0","Type":"ContainerStarted","Data":"796277aa78eebbf5f6bc7da9d60d2551d05b5a1ebbb7f5c7cd54c3a482976b72"}
Sep 17 03:19:53.571932 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.571911    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:19:53.572328 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.572303    2576 generic.go:356] "Generic (PLEG): container finished" podID="ba9064f2-792b-4c2b-b804-5bd64bd69fc5" containerID="bd763ee11454f60c6315612c6e8af5c058d14ba7ac43821f294d54ec7ca54a17" exitCode=1
Sep 17 03:19:53.572444 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.572382    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w969r" event={"ID":"ba9064f2-792b-4c2b-b804-5bd64bd69fc5","Type":"ContainerDied","Data":"bd763ee11454f60c6315612c6e8af5c058d14ba7ac43821f294d54ec7ca54a17"}
Sep 17 03:19:53.572507 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.572490    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w969r" event={"ID":"ba9064f2-792b-4c2b-b804-5bd64bd69fc5","Type":"ContainerStarted","Data":"caf36bf4d4879c6bc9f8b5f2e35b41b83bb5823275ee74a2b14fc1db5f56c014"}
Sep 17 03:19:53.572564 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.572541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w969r" event={"ID":"ba9064f2-792b-4c2b-b804-5bd64bd69fc5","Type":"ContainerStarted","Data":"e6676fe857c2fbd2fa8478a223bb329a08fe735193061e7fb13c70202b6f1609"}
Sep 17 03:19:53.572617 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.572589    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w969r" event={"ID":"ba9064f2-792b-4c2b-b804-5bd64bd69fc5","Type":"ContainerStarted","Data":"c84c9649da27615c32cbd2606546386f78ec6d77e4946d76797e74d9ce707eb6"}
Sep 17 03:19:53.572671 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.572652    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w969r" event={"ID":"ba9064f2-792b-4c2b-b804-5bd64bd69fc5","Type":"ContainerStarted","Data":"c25cb43486013db060d221b2ee53afd65a43b311ea865756c15ed221d400f234"}
Sep 17 03:19:53.574143 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.574123    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f9mkc" event={"ID":"05b2116c-7c2b-4424-8038-a5580cac87b2","Type":"ContainerStarted","Data":"98621e2abe351f43d4d1c95e8d7acbc236631cc55eed95480a2209d2f2bbd3dd"}
Sep 17 03:19:53.575809 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.575784    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-65r5r" event={"ID":"cf8486c3-9f66-4dd8-b244-634629e430fe","Type":"ContainerStarted","Data":"a10fbdd9245bd51378448c79346de3adb63ef967571a5b3e0743e322d124fef2"}
Sep 17 03:19:53.577384 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.577358    2576 generic.go:356] "Generic (PLEG): container finished" podID="8541a873cbf1158c669335dcb651cecd" containerID="0203279599b0d655fc2b36395c416ef06584dd0f5ab32f1cd862c811b76431dd" exitCode=0
Sep 17 03:19:53.577513 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.577462    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal" event={"ID":"8541a873cbf1158c669335dcb651cecd","Type":"ContainerDied","Data":"0203279599b0d655fc2b36395c416ef06584dd0f5ab32f1cd862c811b76431dd"}
Sep 17 03:19:53.592592 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.592544    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-f9mkc" podStartSLOduration=3.045340181 podStartE2EDuration="21.592502339s" totalImagesPullingTime="18.547162158s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:19:53.592371749 +0000 UTC m=+21.937097827" watchObservedRunningTime="2026-09-17 03:19:53.592502339 +0000 UTC m=+21.937228418"
Sep 17 03:19:53.605416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.605369    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-65r5r" podStartSLOduration=6.692527919 podStartE2EDuration="21.605352078s" totalImagesPullingTime="14.912824159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:19:53.604727347 +0000 UTC m=+21.949453425" watchObservedRunningTime="2026-09-17 03:19:53.605352078 +0000 UTC m=+21.950078156"
Sep 17 03:19:53.617127 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.617082    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sp5r2" podStartSLOduration=3.055976155 podStartE2EDuration="21.61706607s" totalImagesPullingTime="18.561089915s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:19:53.616371691 +0000 UTC m=+21.961097772" watchObservedRunningTime="2026-09-17 03:19:53.61706607 +0000 UTC m=+21.961792149"
Sep 17 03:19:53.632504 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.632407    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zxpp2" podStartSLOduration=2.704895138 podStartE2EDuration="21.632391484s" totalImagesPullingTime="18.927496346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:19:53.631741392 +0000 UTC m=+21.976467472" watchObservedRunningTime="2026-09-17 03:19:53.632391484 +0000 UTC m=+21.977117601"
Sep 17 03:19:53.696662 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.696636    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:19:53.763588 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.763549    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:19:53.826817 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.826735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:19:53.829584 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:53.829560    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-65r5r"
Sep 17 03:19:54.287927 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:54.287898    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:54.288296 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:54.288022    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:54.292126 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:54.292106    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 03:19:54.583176 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:54.583140    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n" event={"ID":"f52849ee-79a4-41a0-a215-702f7f31c3a1","Type":"ContainerStarted","Data":"43929c5466924c3a8d93d3b917a371f5df2bdfb1df67be63c854456ba4992a59"}
Sep 17 03:19:54.584881 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:54.584842    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zggrr" event={"ID":"3047fc59-36fa-4bb2-a508-7d1a9a5edf35","Type":"ContainerStarted","Data":"8da5cfe0337eee5e62649843d9f23e194f9d2ed8ed5bd31e6f3635b57e5e5719"}
Sep 17 03:19:54.586959 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:54.586931    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal" event={"ID":"8541a873cbf1158c669335dcb651cecd","Type":"ContainerStarted","Data":"ba7067fa4c2d473e40eba8945a0e0d7ef24f27d25796e0dac527c26f6a4992e1"}
Sep 17 03:19:54.597147 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:54.597093    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zggrr" podStartSLOduration=3.996409576 podStartE2EDuration="22.597073003s" totalImagesPullingTime="18.600663427s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:19:54.596768034 +0000 UTC m=+22.941494113" watchObservedRunningTime="2026-09-17 03:19:54.597073003 +0000 UTC m=+22.941799082"
Sep 17 03:19:54.610476 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:54.610407    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-141.ec2.internal" podStartSLOduration=22.610386593 podStartE2EDuration="22.610386593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:19:54.610360563 +0000 UTC m=+22.955086647" watchObservedRunningTime="2026-09-17 03:19:54.610386593 +0000 UTC m=+22.955112674"
Sep 17 03:19:54.715535 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:54.715506    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:19:54.770628 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:54.770591    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:19:55.203675 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:55.203577    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T03:19:54.292120965Z","UUID":"068eb95e-4d15-4224-ac3d-ca66ea6c6c92","Handler":null,"Name":"","Endpoint":""}
Sep 17 03:19:55.205150 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:55.205123    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 03:19:55.205265 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:55.205160    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 03:19:55.288741 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:55.288707    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:55.288865 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:55.288840    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:55.288958 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:55.288880    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:55.289026 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:55.289006    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:55.591233 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:55.591211    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:19:55.591688 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:55.591649    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w969r" event={"ID":"ba9064f2-792b-4c2b-b804-5bd64bd69fc5","Type":"ContainerStarted","Data":"969304ea4fb2d94642b4ef9d7ab43cb2c40e6748980372335cdf7327ba40a8a5"}
Sep 17 03:19:56.288263 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:56.288230    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:56.288492 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:56.288370    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:56.596876 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:56.596786    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n" event={"ID":"f52849ee-79a4-41a0-a215-702f7f31c3a1","Type":"ContainerStarted","Data":"ed24a3e3fd12c835b901d1c4b2ea1dd69103945debc1df8bd473db836ece51ae"}
Sep 17 03:19:56.612381 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:56.612321    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8d79n" podStartSLOduration=7.4240867680000004 podStartE2EDuration="24.612299608s" totalImagesPullingTime="17.18821284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:19:56.61217612 +0000 UTC m=+24.956902201" watchObservedRunningTime="2026-09-17 03:19:56.612299608 +0000 UTC m=+24.957025688"
Sep 17 03:19:57.288770 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:57.288727    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:57.288770 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:57.288750    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:57.288998 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:57.288868    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:57.289050 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:57.288989    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:58.288388 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:58.288341    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:19:58.288795 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:58.288504    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:19:58.503968 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:58.503693    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-65r5r"
Sep 17 03:19:58.504267 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:58.504247    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-65r5r"
Sep 17 03:19:58.603674 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:58.603638    2576 generic.go:356] "Generic (PLEG): container finished" podID="19037775-9be2-4a9a-aa31-9f1d9fe8aa74" containerID="2104d7c7d43f22d33be94f04669890ed5ce93a93a3555d2e10fb870d21377581" exitCode=0
Sep 17 03:19:58.603848 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:58.603726    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vwpxz" event={"ID":"19037775-9be2-4a9a-aa31-9f1d9fe8aa74","Type":"ContainerDied","Data":"2104d7c7d43f22d33be94f04669890ed5ce93a93a3555d2e10fb870d21377581"}
Sep 17 03:19:58.611239 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:58.611219    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:19:58.611644 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:58.611611    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w969r" event={"ID":"ba9064f2-792b-4c2b-b804-5bd64bd69fc5","Type":"ContainerStarted","Data":"8efaf241049c5aa04e65538fef4dce909a80f474eed7d501cf6e75084a1c1e5b"}
Sep 17 03:19:58.611930 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:58.611910    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:58.612008 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:58.611998    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:58.612074 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:58.612064    2576 scope.go:117] "RemoveContainer" containerID="bd763ee11454f60c6315612c6e8af5c058d14ba7ac43821f294d54ec7ca54a17"
Sep 17 03:19:58.612385 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:58.612367    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-65r5r"
Sep 17 03:19:58.628621 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:58.628599    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:58.720123 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:58.720089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:19:59.288603 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:59.288571    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:19:59.289038 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:59.288572    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:19:59.289038 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:59.288714    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:19:59.289038 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:19:59.288752    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:19:59.521603 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:59.521513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:19:59.581947 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:59.581856    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:19:59.617095 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:59.617068    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vwpxz" event={"ID":"19037775-9be2-4a9a-aa31-9f1d9fe8aa74","Type":"ContainerStarted","Data":"28b15336a93cca2071142f368f059a1fb18b5463d14b8e036fd13057403ec0df"}
Sep 17 03:19:59.621360 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:59.621338    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:19:59.621807 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:59.621772    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w969r" event={"ID":"ba9064f2-792b-4c2b-b804-5bd64bd69fc5","Type":"ContainerStarted","Data":"e2ec03beba18d1da687ef5e0fcee469dcb279350761d2fcc22f1cc2081fef73f"}
Sep 17 03:19:59.621924 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:59.621871    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:59.638184 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:59.638015    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:19:59.658217 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:19:59.658173    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w969r" podStartSLOduration=9.095527688 podStartE2EDuration="27.658157596s" totalImagesPullingTime="18.562629908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:19:59.657752616 +0000 UTC m=+28.002478695" watchObservedRunningTime="2026-09-17 03:19:59.658157596 +0000 UTC m=+28.002883675"
Sep 17 03:20:00.283098 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:00.283066    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9cp4t"]
Sep 17 03:20:00.283317 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:00.283173    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:20:00.283317 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:00.283270    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:20:00.286774 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:00.286746    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b24v9"]
Sep 17 03:20:00.286899 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:00.286838    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:20:00.286966 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:00.286924    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:20:00.288296 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:00.288269    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:20:00.288419 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:00.288397    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:20:00.294125 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:00.294072    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nwswx"]
Sep 17 03:20:00.629212 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:00.629176    2576 generic.go:356] "Generic (PLEG): container finished" podID="19037775-9be2-4a9a-aa31-9f1d9fe8aa74" containerID="28b15336a93cca2071142f368f059a1fb18b5463d14b8e036fd13057403ec0df" exitCode=0
Sep 17 03:20:00.629486 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:00.629262    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:20:00.629486 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:00.629267    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vwpxz" event={"ID":"19037775-9be2-4a9a-aa31-9f1d9fe8aa74","Type":"ContainerDied","Data":"28b15336a93cca2071142f368f059a1fb18b5463d14b8e036fd13057403ec0df"}
Sep 17 03:20:00.629607 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:00.629482    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:20:02.289930 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:02.289618    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:20:02.290310 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:02.289690    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:20:02.290310 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:02.289972    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:20:02.290310 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:02.289715    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:20:02.290310 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:02.290025    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:20:02.290310 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:02.290121    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:20:02.636526 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:02.636490    2576 generic.go:356] "Generic (PLEG): container finished" podID="19037775-9be2-4a9a-aa31-9f1d9fe8aa74" containerID="b3612b47816162778a45856d0f0a193b06ef411102638e4ac07b29e81f0a53dc" exitCode=0
Sep 17 03:20:02.636676 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:02.636574    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vwpxz" event={"ID":"19037775-9be2-4a9a-aa31-9f1d9fe8aa74","Type":"ContainerDied","Data":"b3612b47816162778a45856d0f0a193b06ef411102638e4ac07b29e81f0a53dc"}
Sep 17 03:20:04.288682 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:04.288555    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:20:04.288682 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:04.288589    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:20:04.288682 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:04.288555    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:20:04.289236 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:04.288706    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:20:04.289236 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:04.288787    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:20:04.289236 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:04.288898    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:20:04.765281 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:04.765236    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:04.845136 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:04.845100    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:04.886493 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:04.886464    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs\") pod \"network-metrics-daemon-b24v9\" (UID: \"b21d9f86-427c-4e22-89dd-f30258785552\") " pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:20:04.886673 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:04.886503    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:20:04.886673 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:04.886642    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 03:20:04.886760 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:04.886713    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret podName:2ed49a17-07c3-4f80-8a00-7b5f21fbb61c nodeName:}" failed. No retries permitted until 2026-09-17 03:20:36.886691977 +0000 UTC m=+65.231418041 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret") pod "global-pull-secret-syncer-9cp4t" (UID: "2ed49a17-07c3-4f80-8a00-7b5f21fbb61c") : object "kube-system"/"original-pull-secret" not registered
Sep 17 03:20:04.886760 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:04.886644    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:20:04.886860 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:04.886807    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs podName:b21d9f86-427c-4e22-89dd-f30258785552 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:36.8867877 +0000 UTC m=+65.231513764 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs") pod "network-metrics-daemon-b24v9" (UID: "b21d9f86-427c-4e22-89dd-f30258785552") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 03:20:04.987310 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:04.987273    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64f9j\" (UniqueName: \"kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j\") pod \"network-check-target-nwswx\" (UID: \"6e66fbb6-8a12-4388-9b2b-4f4d256673cc\") " pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:20:04.987514 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:04.987477    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 03:20:04.987514 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:04.987501    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 03:20:04.987514 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:04.987514    2576 projected.go:196] Error preparing data for projected volume kube-api-access-64f9j for pod openshift-network-diagnostics/network-check-target-nwswx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:20:04.987676 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:04.987579    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j podName:6e66fbb6-8a12-4388-9b2b-4f4d256673cc nodeName:}" failed. No retries permitted until 2026-09-17 03:20:36.987560408 +0000 UTC m=+65.332286468 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-64f9j" (UniqueName: "kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j") pod "network-check-target-nwswx" (UID: "6e66fbb6-8a12-4388-9b2b-4f4d256673cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 03:20:05.208316 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:05.208281    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:05.343086 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:05.343056    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:06.288221 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.288127    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:20:06.288499 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.288264    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:20:06.288499 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:06.288267    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9cp4t" podUID="2ed49a17-07c3-4f80-8a00-7b5f21fbb61c"
Sep 17 03:20:06.288499 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.288284    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:20:06.288499 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:06.288367    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nwswx" podUID="6e66fbb6-8a12-4388-9b2b-4f4d256673cc"
Sep 17 03:20:06.288499 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:06.288481    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b24v9" podUID="b21d9f86-427c-4e22-89dd-f30258785552"
Sep 17 03:20:06.440567 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.440531    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:06.457566 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.457533    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-141.ec2.internal" event="NodeReady"
Sep 17 03:20:06.457742 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.457697    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 03:20:06.499259 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.499223    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7hqg9"]
Sep 17 03:20:06.507773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.507741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:06.532695 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.532645    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-44gmz"]
Sep 17 03:20:06.532906 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.532788    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:06.534813 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.534789    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8xsvg\""
Sep 17 03:20:06.535047 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.535028    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 03:20:06.535201 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.535182    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 03:20:06.546042 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.546014    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7hqg9"]
Sep 17 03:20:06.546161 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.546064    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-44gmz"]
Sep 17 03:20:06.546161 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.546127    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-44gmz"
Sep 17 03:20:06.548266 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.548243    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 03:20:06.548411 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.548296    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 03:20:06.548540 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.548461    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tddkt\""
Sep 17 03:20:06.548779 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.548762    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 03:20:06.601376 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.601345    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-tmp-dir\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:06.601569 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.601444    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-config-volume\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:06.601569 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.601494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:06.601569 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.601516    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw28l\" (UniqueName: \"kubernetes.io/projected/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-kube-api-access-fw28l\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:06.614753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.614715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:06.666374 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.666340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:06.702421 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.702375    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-tmp-dir\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:06.702620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.702445    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwwsq\" (UniqueName: \"kubernetes.io/projected/fcc07e33-2671-4c55-9211-9714c8799a6e-kube-api-access-xwwsq\") pod \"ingress-canary-44gmz\" (UID: \"fcc07e33-2671-4c55-9211-9714c8799a6e\") " pod="openshift-ingress-canary/ingress-canary-44gmz"
Sep 17 03:20:06.702620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.702502    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-config-volume\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:06.702620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.702545    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:06.702620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.702565    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fcc07e33-2671-4c55-9211-9714c8799a6e-cert\") pod \"ingress-canary-44gmz\" (UID: \"fcc07e33-2671-4c55-9211-9714c8799a6e\") " pod="openshift-ingress-canary/ingress-canary-44gmz"
Sep 17 03:20:06.702620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.702592    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw28l\" (UniqueName: \"kubernetes.io/projected/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-kube-api-access-fw28l\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:06.702843 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.702825    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-tmp-dir\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:06.702945 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:06.702929    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 03:20:06.703002 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:06.702991    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls podName:b20e378f-f35f-420a-9b02-fd76bf5a2bd1 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:07.202972908 +0000 UTC m=+35.547698982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls") pod "dns-default-7hqg9" (UID: "b20e378f-f35f-420a-9b02-fd76bf5a2bd1") : secret "dns-default-metrics-tls" not found
Sep 17 03:20:06.703064 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.703031    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-config-volume\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:06.712957 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.712924    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw28l\" (UniqueName: \"kubernetes.io/projected/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-kube-api-access-fw28l\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:06.803949 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.803909    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwwsq\" (UniqueName: \"kubernetes.io/projected/fcc07e33-2671-4c55-9211-9714c8799a6e-kube-api-access-xwwsq\") pod \"ingress-canary-44gmz\" (UID: \"fcc07e33-2671-4c55-9211-9714c8799a6e\") " pod="openshift-ingress-canary/ingress-canary-44gmz"
Sep 17 03:20:06.804143 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.804019    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fcc07e33-2671-4c55-9211-9714c8799a6e-cert\") pod \"ingress-canary-44gmz\" (UID: \"fcc07e33-2671-4c55-9211-9714c8799a6e\") " pod="openshift-ingress-canary/ingress-canary-44gmz"
Sep 17 03:20:06.806947 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.806919    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fcc07e33-2671-4c55-9211-9714c8799a6e-cert\") pod \"ingress-canary-44gmz\" (UID: \"fcc07e33-2671-4c55-9211-9714c8799a6e\") " pod="openshift-ingress-canary/ingress-canary-44gmz"
Sep 17 03:20:06.812351 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.812322    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwwsq\" (UniqueName: \"kubernetes.io/projected/fcc07e33-2671-4c55-9211-9714c8799a6e-kube-api-access-xwwsq\") pod \"ingress-canary-44gmz\" (UID: \"fcc07e33-2671-4c55-9211-9714c8799a6e\") " pod="openshift-ingress-canary/ingress-canary-44gmz"
Sep 17 03:20:06.855396 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:06.855362    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-44gmz"
Sep 17 03:20:07.208055 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:07.207824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:07.208055 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:07.207969    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 03:20:07.208238 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:07.208091    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls podName:b20e378f-f35f-420a-9b02-fd76bf5a2bd1 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:08.208075725 +0000 UTC m=+36.552801782 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls") pod "dns-default-7hqg9" (UID: "b20e378f-f35f-420a-9b02-fd76bf5a2bd1") : secret "dns-default-metrics-tls" not found
Sep 17 03:20:07.235945 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:07.235644    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-44gmz"]
Sep 17 03:20:07.239668 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:20:07.239647    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcc07e33_2671_4c55_9211_9714c8799a6e.slice/crio-1c9feaeb7e7a5fbd6983b04341cdc345fe2ad9737db6eb154a6a7e9439ea1bd5 WatchSource:0}: Error finding container 1c9feaeb7e7a5fbd6983b04341cdc345fe2ad9737db6eb154a6a7e9439ea1bd5: Status 404 returned error can't find the container with id 1c9feaeb7e7a5fbd6983b04341cdc345fe2ad9737db6eb154a6a7e9439ea1bd5
Sep 17 03:20:07.651289 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:07.651254    2576 generic.go:356] "Generic (PLEG): container finished" podID="19037775-9be2-4a9a-aa31-9f1d9fe8aa74" containerID="b087979634906feef7e38730e0d0a6bc3e8a76a0ca4d267a9952dd210647edea" exitCode=0
Sep 17 03:20:07.651728 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:07.651333    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vwpxz" event={"ID":"19037775-9be2-4a9a-aa31-9f1d9fe8aa74","Type":"ContainerDied","Data":"b087979634906feef7e38730e0d0a6bc3e8a76a0ca4d267a9952dd210647edea"}
Sep 17 03:20:07.652532 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:07.652503    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-44gmz" event={"ID":"fcc07e33-2671-4c55-9211-9714c8799a6e","Type":"ContainerStarted","Data":"1c9feaeb7e7a5fbd6983b04341cdc345fe2ad9737db6eb154a6a7e9439ea1bd5"}
Sep 17 03:20:08.214644 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:08.214607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:08.214824 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:08.214756    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 03:20:08.214872 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:08.214830    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls podName:b20e378f-f35f-420a-9b02-fd76bf5a2bd1 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:10.214805263 +0000 UTC m=+38.559531320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls") pod "dns-default-7hqg9" (UID: "b20e378f-f35f-420a-9b02-fd76bf5a2bd1") : secret "dns-default-metrics-tls" not found
Sep 17 03:20:08.288836 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:08.288798    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:20:08.289015 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:08.288798    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:20:08.289015 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:08.288917    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:20:08.291552 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:08.291524    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d2k6b\""
Sep 17 03:20:08.291552 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:08.291542    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 03:20:08.291753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:08.291528    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tmrnq\""
Sep 17 03:20:08.291753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:08.291575    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 03:20:08.291753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:08.291623    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 03:20:08.291753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:08.291524    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 03:20:08.658244 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:08.658215    2576 generic.go:356] "Generic (PLEG): container finished" podID="19037775-9be2-4a9a-aa31-9f1d9fe8aa74" containerID="f2647c4aaf23474203754c40932507580c6fd8c9cb1e9579ba331ff596f8df7e" exitCode=0
Sep 17 03:20:08.658811 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:08.658259    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vwpxz" event={"ID":"19037775-9be2-4a9a-aa31-9f1d9fe8aa74","Type":"ContainerDied","Data":"f2647c4aaf23474203754c40932507580c6fd8c9cb1e9579ba331ff596f8df7e"}
Sep 17 03:20:09.665408 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:09.665374    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vwpxz" event={"ID":"19037775-9be2-4a9a-aa31-9f1d9fe8aa74","Type":"ContainerStarted","Data":"d2d6cc9f4dea59b37c8dc448b81a16903881a6ea8d613a405147f89b9134e4a4"}
Sep 17 03:20:09.684642 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:09.684510    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vwpxz" podStartSLOduration=7.882911039 podStartE2EDuration="37.684490329s" totalImagesPullingTime="29.80157929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:20:09.683308948 +0000 UTC m=+38.028035027" watchObservedRunningTime="2026-09-17 03:20:09.684490329 +0000 UTC m=+38.029216409"
Sep 17 03:20:09.830244 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:09.830212    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:09.879745 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:09.879726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:10.229034 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:10.228944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:10.229218 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:10.229132    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 03:20:10.229218 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:10.229210    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls podName:b20e378f-f35f-420a-9b02-fd76bf5a2bd1 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:14.229190756 +0000 UTC m=+42.573916826 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls") pod "dns-default-7hqg9" (UID: "b20e378f-f35f-420a-9b02-fd76bf5a2bd1") : secret "dns-default-metrics-tls" not found
Sep 17 03:20:10.234343 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:10.234315    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:10.335373 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:10.335339    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:10.756638 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:10.756608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:10.847222 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:10.847192    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:10.947344 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:10.947309    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:11.004259 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:11.004225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:11.465748 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:11.465701    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:11.548501 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:11.548469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:11.902183 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:11.902137    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:11.904843 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:11.904818    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nxtgf"]
Sep 17 03:20:11.927253 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:11.927231    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nxtgf"]
Sep 17 03:20:11.927363 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:11.927342    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:11.929253 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:11.929225    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 03:20:11.929371 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:11.929274    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 03:20:11.929371 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:11.929280    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 03:20:11.929371 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:11.929330    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-52cf6\""
Sep 17 03:20:11.929512 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:11.929412    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 03:20:11.966155 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:11.966121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:12.043340 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.043297    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/40fe1061-8f7b-4385-9c4f-75311f202ac7-crio-socket\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.043538 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.043351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/40fe1061-8f7b-4385-9c4f-75311f202ac7-data-volume\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.043538 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.043371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwrh7\" (UniqueName: \"kubernetes.io/projected/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-api-access-fwrh7\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.043538 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.043438    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.043538 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.043467    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.143804 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.143763    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.143996 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:12.143911    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 03:20:12.143996 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.143926    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/40fe1061-8f7b-4385-9c4f-75311f202ac7-crio-socket\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.143996 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.143961    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/40fe1061-8f7b-4385-9c4f-75311f202ac7-data-volume\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.143996 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:12.143973    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls podName:40fe1061-8f7b-4385-9c4f-75311f202ac7 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:12.643956871 +0000 UTC m=+40.988682929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nxtgf" (UID: "40fe1061-8f7b-4385-9c4f-75311f202ac7") : secret "insights-runtime-extractor-tls" not found
Sep 17 03:20:12.144211 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.144006    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwrh7\" (UniqueName: \"kubernetes.io/projected/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-api-access-fwrh7\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.144211 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.144074    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.144211 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:12.144153    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 03:20:12.144211 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:12.144203    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm podName:40fe1061-8f7b-4385-9c4f-75311f202ac7 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:12.644190777 +0000 UTC m=+40.988916853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nxtgf" (UID: "40fe1061-8f7b-4385-9c4f-75311f202ac7") : configmap "kube-rbac-proxy" not found
Sep 17 03:20:12.144211 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.144206    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/40fe1061-8f7b-4385-9c4f-75311f202ac7-crio-socket\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.144402 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.144283    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/40fe1061-8f7b-4385-9c4f-75311f202ac7-data-volume\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.154087 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.154019    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-qv9xm"]
Sep 17 03:20:12.155447 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.155408    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:12.158712 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.158696    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwrh7\" (UniqueName: \"kubernetes.io/projected/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-api-access-fwrh7\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.175097 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.175066    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-qv9xm"]
Sep 17 03:20:12.175228 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.175174    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qv9xm"
Sep 17 03:20:12.177341 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.177320    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 17 03:20:12.177551 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.177532    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-dmft7\""
Sep 17 03:20:12.177620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.177586    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 17 03:20:12.213423 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.213385    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:12.244527 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.244493    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7bt6\" (UniqueName: \"kubernetes.io/projected/bbc0d1a4-1a90-4ee8-b668-9e7802ed999c-kube-api-access-h7bt6\") pod \"migrator-8b4d697c5-qv9xm\" (UID: \"bbc0d1a4-1a90-4ee8-b668-9e7802ed999c\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qv9xm"
Sep 17 03:20:12.345587 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.345547    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7bt6\" (UniqueName: \"kubernetes.io/projected/bbc0d1a4-1a90-4ee8-b668-9e7802ed999c-kube-api-access-h7bt6\") pod \"migrator-8b4d697c5-qv9xm\" (UID: \"bbc0d1a4-1a90-4ee8-b668-9e7802ed999c\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qv9xm"
Sep 17 03:20:12.353763 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.353726    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7bt6\" (UniqueName: \"kubernetes.io/projected/bbc0d1a4-1a90-4ee8-b668-9e7802ed999c-kube-api-access-h7bt6\") pod \"migrator-8b4d697c5-qv9xm\" (UID: \"bbc0d1a4-1a90-4ee8-b668-9e7802ed999c\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qv9xm"
Sep 17 03:20:12.484175 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.484088    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qv9xm"
Sep 17 03:20:12.649400 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.649356    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.649589 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.649409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:12.649589 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:12.649497    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 03:20:12.649589 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:12.649563    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm podName:40fe1061-8f7b-4385-9c4f-75311f202ac7 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:13.64954841 +0000 UTC m=+41.994274466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nxtgf" (UID: "40fe1061-8f7b-4385-9c4f-75311f202ac7") : configmap "kube-rbac-proxy" not found
Sep 17 03:20:12.649780 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:12.649609    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 03:20:12.649780 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:12.649673    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls podName:40fe1061-8f7b-4385-9c4f-75311f202ac7 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:13.649658884 +0000 UTC m=+41.994384941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nxtgf" (UID: "40fe1061-8f7b-4385-9c4f-75311f202ac7") : secret "insights-runtime-extractor-tls" not found
Sep 17 03:20:12.767815 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:12.767731    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-qv9xm"]
Sep 17 03:20:13.107741 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:13.107700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:13.177622 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:13.177590    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:13.377915 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:20:13.377835    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc0d1a4_1a90_4ee8_b668_9e7802ed999c.slice/crio-94c360344a5aa1a74a9133772f61dccffac3c2bdf2bf5d482a84086935919505 WatchSource:0}: Error finding container 94c360344a5aa1a74a9133772f61dccffac3c2bdf2bf5d482a84086935919505: Status 404 returned error can't find the container with id 94c360344a5aa1a74a9133772f61dccffac3c2bdf2bf5d482a84086935919505
Sep 17 03:20:13.657089 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:13.657012    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:13.657089 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:13.657055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:13.657272 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:13.657137    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 03:20:13.657272 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:13.657201    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm podName:40fe1061-8f7b-4385-9c4f-75311f202ac7 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:15.657186077 +0000 UTC m=+44.001912141 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nxtgf" (UID: "40fe1061-8f7b-4385-9c4f-75311f202ac7") : configmap "kube-rbac-proxy" not found
Sep 17 03:20:13.657272 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:13.657237    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 03:20:13.657375 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:13.657305    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls podName:40fe1061-8f7b-4385-9c4f-75311f202ac7 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:15.657290508 +0000 UTC m=+44.002016587 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nxtgf" (UID: "40fe1061-8f7b-4385-9c4f-75311f202ac7") : secret "insights-runtime-extractor-tls" not found
Sep 17 03:20:13.676471 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:13.676417    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qv9xm" event={"ID":"bbc0d1a4-1a90-4ee8-b668-9e7802ed999c","Type":"ContainerStarted","Data":"94c360344a5aa1a74a9133772f61dccffac3c2bdf2bf5d482a84086935919505"}
Sep 17 03:20:13.867008 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:13.866980    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f9mkc_05b2116c-7c2b-4424-8038-a5580cac87b2/dns-node-resolver/0.log"
Sep 17 03:20:14.024177 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:14.024086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:14.262319 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:14.262282    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:14.262773 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:14.262475    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 03:20:14.262773 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:14.262552    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls podName:b20e378f-f35f-420a-9b02-fd76bf5a2bd1 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:22.262531675 +0000 UTC m=+50.607257756 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls") pod "dns-default-7hqg9" (UID: "b20e378f-f35f-420a-9b02-fd76bf5a2bd1") : secret "dns-default-metrics-tls" not found
Sep 17 03:20:14.680609 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:14.680569    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-44gmz" event={"ID":"fcc07e33-2671-4c55-9211-9714c8799a6e","Type":"ContainerStarted","Data":"f09be706409f13939b8b665c054e9a7654522454f893998726b2830ea02d2a64"}
Sep 17 03:20:14.697241 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:14.697188    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-44gmz" podStartSLOduration=2.244079241 podStartE2EDuration="8.697168146s" totalImagesPullingTime="6.453088905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:20:14.69581174 +0000 UTC m=+43.040537821" watchObservedRunningTime="2026-09-17 03:20:14.697168146 +0000 UTC m=+43.041894331"
Sep 17 03:20:15.070938 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:15.070910    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sp5r2_7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158/node-ca/0.log"
Sep 17 03:20:15.101982 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:15.101943    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:15.202289 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:15.202254    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:15.303729 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:15.303698    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:15.403181 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:15.403049    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:15.465947 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:15.465921    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-44gmz_fcc07e33-2671-4c55-9211-9714c8799a6e/serve-healthcheck-canary/0.log"
Sep 17 03:20:15.503106 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:15.503068    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:15.673912 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:15.673808    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:15.673912 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:15.673911    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:15.674094 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:15.673962    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 03:20:15.674094 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:15.673981    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 03:20:15.674094 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:15.674027    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls podName:40fe1061-8f7b-4385-9c4f-75311f202ac7 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:19.674011586 +0000 UTC m=+48.018737643 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nxtgf" (UID: "40fe1061-8f7b-4385-9c4f-75311f202ac7") : secret "insights-runtime-extractor-tls" not found
Sep 17 03:20:15.674094 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:15.674042    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm podName:40fe1061-8f7b-4385-9c4f-75311f202ac7 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:19.674035974 +0000 UTC m=+48.018762031 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nxtgf" (UID: "40fe1061-8f7b-4385-9c4f-75311f202ac7") : configmap "kube-rbac-proxy" not found
Sep 17 03:20:15.685466 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:15.685419    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qv9xm" event={"ID":"bbc0d1a4-1a90-4ee8-b668-9e7802ed999c","Type":"ContainerStarted","Data":"0dc3ba87efb35da065a85d9c0201c5102a78e4b0731997aedffa53de0cad5602"}
Sep 17 03:20:15.685635 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:15.685510    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qv9xm" event={"ID":"bbc0d1a4-1a90-4ee8-b668-9e7802ed999c","Type":"ContainerStarted","Data":"6de8f41f5c0ffe57db5bf095c70ac178094ba9782539bf329092873d0766465d"}
Sep 17 03:20:15.705365 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:15.705311    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qv9xm" podStartSLOduration=1.774107235 podStartE2EDuration="3.705294999s" totalImagesPullingTime="1.931187764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:20:15.70484752 +0000 UTC m=+44.049573601" watchObservedRunningTime="2026-09-17 03:20:15.705294999 +0000 UTC m=+44.050021107"
Sep 17 03:20:16.490157 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:16.490122    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:16.792879 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:16.792800    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:16.993220 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:16.993192    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:17.195955 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:17.195922    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:17.252653 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:17.252625    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:17.591522 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:17.591492    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:18.391119 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:18.391089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:18.594160 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:18.594132    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:18.797689 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:18.797600    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:18.990705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:18.990668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:19.041036 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:19.041008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:19.195754 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:19.195724    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:19.593013 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:19.592979    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:19.694832 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:19.694792    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:19.704861 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:19.704834    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:19.705023 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:19.704988    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 03:20:19.705023 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:19.705014    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:19.705130 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:19.705067    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls podName:40fe1061-8f7b-4385-9c4f-75311f202ac7 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:27.70504721 +0000 UTC m=+56.049773268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nxtgf" (UID: "40fe1061-8f7b-4385-9c4f-75311f202ac7") : secret "insights-runtime-extractor-tls" not found
Sep 17 03:20:19.705130 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:19.705091    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 03:20:19.705226 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:19.705146    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm podName:40fe1061-8f7b-4385-9c4f-75311f202ac7 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:27.705132205 +0000 UTC m=+56.049858263 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nxtgf" (UID: "40fe1061-8f7b-4385-9c4f-75311f202ac7") : configmap "kube-rbac-proxy" not found
Sep 17 03:20:19.989398 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:19.989309    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:20.191855 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:20.191827    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:21.354194 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:21.354153    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:21.408568 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:21.408522    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:21.601126 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:21.601087    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:21.788456 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:21.788353    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:21.994799 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:21.994765    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:22.192408 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:22.192376    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:22.325228 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:22.325190    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:22.327464 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:22.327442    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b20e378f-f35f-420a-9b02-fd76bf5a2bd1-metrics-tls\") pod \"dns-default-7hqg9\" (UID: \"b20e378f-f35f-420a-9b02-fd76bf5a2bd1\") " pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:22.445024 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:22.444938    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:22.709008 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:22.708934    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7hqg9"]
Sep 17 03:20:22.712384 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:20:22.712357    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb20e378f_f35f_420a_9b02_fd76bf5a2bd1.slice/crio-b91f237eb063f1232e893b7af15a90cf3f1850311ae673cddb7bfea5ae3748be WatchSource:0}: Error finding container b91f237eb063f1232e893b7af15a90cf3f1850311ae673cddb7bfea5ae3748be: Status 404 returned error can't find the container with id b91f237eb063f1232e893b7af15a90cf3f1850311ae673cddb7bfea5ae3748be
Sep 17 03:20:23.193988 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:23.193955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:23.594050 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:23.594011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:23.707380 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:23.707330    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7hqg9" event={"ID":"b20e378f-f35f-420a-9b02-fd76bf5a2bd1","Type":"ContainerStarted","Data":"b91f237eb063f1232e893b7af15a90cf3f1850311ae673cddb7bfea5ae3748be"}
Sep 17 03:20:23.832323 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:23.832282    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:24.015031 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:24.014941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:24.094840 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:24.094807    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:24.202649 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:24.202599    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:24.713445 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:24.713410    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7hqg9" event={"ID":"b20e378f-f35f-420a-9b02-fd76bf5a2bd1","Type":"ContainerStarted","Data":"07a8097ad04198919882447871048ee7f98e5ebfc1db7113ddcc0cc54963f575"}
Sep 17 03:20:24.713849 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:24.713479    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7hqg9" event={"ID":"b20e378f-f35f-420a-9b02-fd76bf5a2bd1","Type":"ContainerStarted","Data":"04c743df2f07be0258d401c4b37bc9444141d6ced4778f29200e1287d11b6698"}
Sep 17 03:20:24.713849 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:24.713592    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:24.735358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:24.735308    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7hqg9" podStartSLOduration=16.909928817 podStartE2EDuration="18.735292328s" totalImagesPullingTime="1.825363511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:20:24.735036907 +0000 UTC m=+53.079763019" watchObservedRunningTime="2026-09-17 03:20:24.735292328 +0000 UTC m=+53.080018442"
Sep 17 03:20:25.009952 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:25.009906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:25.394816 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:25.394773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:25.593151 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:25.593121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:25.647462 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:25.647367    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:25.799779 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:25.799746    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:27.766278 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:27.766238    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:27.766278 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:27.766285    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:27.766962 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:27.766939    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/40fe1061-8f7b-4385-9c4f-75311f202ac7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:27.768578 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:27.768560    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/40fe1061-8f7b-4385-9c4f-75311f202ac7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nxtgf\" (UID: \"40fe1061-8f7b-4385-9c4f-75311f202ac7\") " pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:27.836261 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:27.836224    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nxtgf"
Sep 17 03:20:28.106306 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:28.106273    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nxtgf"]
Sep 17 03:20:28.110343 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:20:28.110310    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40fe1061_8f7b_4385_9c4f_75311f202ac7.slice/crio-270839b4443294078f2ab11fae8c50d52510224f17092fc6a2b313170b77442c WatchSource:0}: Error finding container 270839b4443294078f2ab11fae8c50d52510224f17092fc6a2b313170b77442c: Status 404 returned error can't find the container with id 270839b4443294078f2ab11fae8c50d52510224f17092fc6a2b313170b77442c
Sep 17 03:20:28.727710 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:28.727674    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nxtgf" event={"ID":"40fe1061-8f7b-4385-9c4f-75311f202ac7","Type":"ContainerStarted","Data":"933a9a92eb514ceb47f0285a32a4978e3085e615a29c101c79dfdbfd8bff6a80"}
Sep 17 03:20:28.727879 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:28.727741    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nxtgf" event={"ID":"40fe1061-8f7b-4385-9c4f-75311f202ac7","Type":"ContainerStarted","Data":"270839b4443294078f2ab11fae8c50d52510224f17092fc6a2b313170b77442c"}
Sep 17 03:20:29.733214 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:29.733179    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nxtgf" event={"ID":"40fe1061-8f7b-4385-9c4f-75311f202ac7","Type":"ContainerStarted","Data":"2f60dbee06152aa40b503591a878224060631dbc2a20c786a5c34452dbeb0955"}
Sep 17 03:20:30.738683 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:30.738582    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nxtgf" event={"ID":"40fe1061-8f7b-4385-9c4f-75311f202ac7","Type":"ContainerStarted","Data":"b53a8ea61dcc513ec44ee8a7c914ee199d9c535455b9d3b188a84170058bf4d3"}
Sep 17 03:20:30.754953 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:30.754902    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nxtgf" podStartSLOduration=17.601578622 podStartE2EDuration="19.754887765s" totalImagesPullingTime="2.153309143s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 03:20:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:20:30.754391726 +0000 UTC m=+59.099117806" watchObservedRunningTime="2026-09-17 03:20:30.754887765 +0000 UTC m=+59.099613843"
Sep 17 03:20:31.646195 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:31.646167    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w969r"
Sep 17 03:20:33.887879 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.887844    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h"]
Sep 17 03:20:33.890550 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.890533    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h"
Sep 17 03:20:33.893159 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.893134    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 17 03:20:33.893316 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.893167    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 17 03:20:33.893316 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.893174    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 17 03:20:33.893316 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.893134    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 17 03:20:33.893316 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.893140    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4ztwl\""
Sep 17 03:20:33.896230 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.896207    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"]
Sep 17 03:20:33.898494 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.898472    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:33.900508 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.900485    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 17 03:20:33.900508 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.900506    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 17 03:20:33.900770 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.900753    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 17 03:20:33.900965 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.900951    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 17 03:20:33.902766 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.902744    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h"]
Sep 17 03:20:33.910420 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.910382    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/39eef97b-24d1-41dd-b78e-d0efe143494f-ca\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:33.910525 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.910457    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/39eef97b-24d1-41dd-b78e-d0efe143494f-hub\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:33.910525 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.910507    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/39eef97b-24d1-41dd-b78e-d0efe143494f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:33.910602 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.910543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/39eef97b-24d1-41dd-b78e-d0efe143494f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:33.910718 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.910698    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/39eef97b-24d1-41dd-b78e-d0efe143494f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:33.918102 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.918077    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"]
Sep 17 03:20:33.998492 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:33.998458    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:34.011980 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.011947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/39eef97b-24d1-41dd-b78e-d0efe143494f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.012125 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.011985    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfa820e0-5da0-43c4-86b6-9a80c393fb7a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86d5f77c96-4mm2h\" (UID: \"dfa820e0-5da0-43c4-86b6-9a80c393fb7a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h"
Sep 17 03:20:34.012125 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.012008    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn6lv\" (UniqueName: \"kubernetes.io/projected/dfa820e0-5da0-43c4-86b6-9a80c393fb7a-kube-api-access-jn6lv\") pod \"managed-serviceaccount-addon-agent-86d5f77c96-4mm2h\" (UID: \"dfa820e0-5da0-43c4-86b6-9a80c393fb7a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h"
Sep 17 03:20:34.012125 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.012043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/39eef97b-24d1-41dd-b78e-d0efe143494f-ca\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.012125 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.012059    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfdmk\" (UniqueName: \"kubernetes.io/projected/39eef97b-24d1-41dd-b78e-d0efe143494f-kube-api-access-dfdmk\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.012125 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.012083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/39eef97b-24d1-41dd-b78e-d0efe143494f-hub\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.012125 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.012099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/39eef97b-24d1-41dd-b78e-d0efe143494f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.012125 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.012119    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/39eef97b-24d1-41dd-b78e-d0efe143494f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.012942 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.012912    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/39eef97b-24d1-41dd-b78e-d0efe143494f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.014572 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.014545    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/39eef97b-24d1-41dd-b78e-d0efe143494f-hub\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.014684 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.014617    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/39eef97b-24d1-41dd-b78e-d0efe143494f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.014684 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.014631    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/39eef97b-24d1-41dd-b78e-d0efe143494f-ca\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.014796 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.014698    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/39eef97b-24d1-41dd-b78e-d0efe143494f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.098037 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.098005    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:34.112819 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.112793    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfa820e0-5da0-43c4-86b6-9a80c393fb7a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86d5f77c96-4mm2h\" (UID: \"dfa820e0-5da0-43c4-86b6-9a80c393fb7a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h"
Sep 17 03:20:34.112896 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.112824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn6lv\" (UniqueName: \"kubernetes.io/projected/dfa820e0-5da0-43c4-86b6-9a80c393fb7a-kube-api-access-jn6lv\") pod \"managed-serviceaccount-addon-agent-86d5f77c96-4mm2h\" (UID: \"dfa820e0-5da0-43c4-86b6-9a80c393fb7a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h"
Sep 17 03:20:34.112896 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.112855    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfdmk\" (UniqueName: \"kubernetes.io/projected/39eef97b-24d1-41dd-b78e-d0efe143494f-kube-api-access-dfdmk\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.115290 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.115269    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfa820e0-5da0-43c4-86b6-9a80c393fb7a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-86d5f77c96-4mm2h\" (UID: \"dfa820e0-5da0-43c4-86b6-9a80c393fb7a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h"
Sep 17 03:20:34.127193 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.127161    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn6lv\" (UniqueName: \"kubernetes.io/projected/dfa820e0-5da0-43c4-86b6-9a80c393fb7a-kube-api-access-jn6lv\") pod \"managed-serviceaccount-addon-agent-86d5f77c96-4mm2h\" (UID: \"dfa820e0-5da0-43c4-86b6-9a80c393fb7a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h"
Sep 17 03:20:34.127193 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.127179    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfdmk\" (UniqueName: \"kubernetes.io/projected/39eef97b-24d1-41dd-b78e-d0efe143494f-kube-api-access-dfdmk\") pod \"cluster-proxy-proxy-agent-856444975-plsrs\" (UID: \"39eef97b-24d1-41dd-b78e-d0efe143494f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.196074 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.195981    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:34.205944 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.205908    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h"
Sep 17 03:20:34.213320 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.213300    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"
Sep 17 03:20:34.295412 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.295385    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:34.395880 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.395846    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:34.449994 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.449923    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:34.471936 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.471904    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h"]
Sep 17 03:20:34.474928 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:20:34.474898    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfa820e0_5da0_43c4_86b6_9a80c393fb7a.slice/crio-feae01fe66eeaac93cc55e2e2c8005242c9e146ae0e49b081742cb872d0bcc17 WatchSource:0}: Error finding container feae01fe66eeaac93cc55e2e2c8005242c9e146ae0e49b081742cb872d0bcc17: Status 404 returned error can't find the container with id feae01fe66eeaac93cc55e2e2c8005242c9e146ae0e49b081742cb872d0bcc17
Sep 17 03:20:34.486084 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.486024    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs"]
Sep 17 03:20:34.488580 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:20:34.488557    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39eef97b_24d1_41dd_b78e_d0efe143494f.slice/crio-660b0d900e0ed33ef26fbde61c39ea583864dc9a8416f92ee67ebfb519df52d7 WatchSource:0}: Error finding container 660b0d900e0ed33ef26fbde61c39ea583864dc9a8416f92ee67ebfb519df52d7: Status 404 returned error can't find the container with id 660b0d900e0ed33ef26fbde61c39ea583864dc9a8416f92ee67ebfb519df52d7
Sep 17 03:20:34.718495 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.718399    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7hqg9"
Sep 17 03:20:34.752502 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.752469    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h" event={"ID":"dfa820e0-5da0-43c4-86b6-9a80c393fb7a","Type":"ContainerStarted","Data":"feae01fe66eeaac93cc55e2e2c8005242c9e146ae0e49b081742cb872d0bcc17"}
Sep 17 03:20:34.753586 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:34.753560    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs" event={"ID":"39eef97b-24d1-41dd-b78e-d0efe143494f","Type":"ContainerStarted","Data":"660b0d900e0ed33ef26fbde61c39ea583864dc9a8416f92ee67ebfb519df52d7"}
Sep 17 03:20:36.931812 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:36.931769    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs\") pod \"network-metrics-daemon-b24v9\" (UID: \"b21d9f86-427c-4e22-89dd-f30258785552\") " pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:20:36.932302 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:36.931833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:20:36.933893 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:36.933854    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 03:20:36.934024 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:36.933897    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 03:20:36.945450 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:36.945368    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ed49a17-07c3-4f80-8a00-7b5f21fbb61c-original-pull-secret\") pod \"global-pull-secret-syncer-9cp4t\" (UID: \"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c\") " pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:20:36.945582 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:36.945545    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b21d9f86-427c-4e22-89dd-f30258785552-metrics-certs\") pod \"network-metrics-daemon-b24v9\" (UID: \"b21d9f86-427c-4e22-89dd-f30258785552\") " pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:20:36.969671 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:36.969632    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:37.021688 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:37.021644    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:37.032830 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:37.032799    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64f9j\" (UniqueName: \"kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j\") pod \"network-check-target-nwswx\" (UID: \"6e66fbb6-8a12-4388-9b2b-4f4d256673cc\") " pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:20:37.034660 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:37.034638    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 03:20:37.044847 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:37.044820    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 03:20:37.056173 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:37.056142    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64f9j\" (UniqueName: \"kubernetes.io/projected/6e66fbb6-8a12-4388-9b2b-4f4d256673cc-kube-api-access-64f9j\") pod \"network-check-target-nwswx\" (UID: \"6e66fbb6-8a12-4388-9b2b-4f4d256673cc\") " pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:20:37.098922 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:37.098881    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9cp4t"
Sep 17 03:20:37.106792 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:37.106767    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d2k6b\""
Sep 17 03:20:37.112861 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:37.112835    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tmrnq\""
Sep 17 03:20:37.115317 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:37.115294    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:20:37.120122 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:37.120103    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b24v9"
Sep 17 03:20:38.582169 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:38.582140    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nwswx"]
Sep 17 03:20:38.585267 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:20:38.585240    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e66fbb6_8a12_4388_9b2b_4f4d256673cc.slice/crio-d48f8544343f9a483ce236f4c107fc45baad66a5f223832afab49b306a0189b0 WatchSource:0}: Error finding container d48f8544343f9a483ce236f4c107fc45baad66a5f223832afab49b306a0189b0: Status 404 returned error can't find the container with id d48f8544343f9a483ce236f4c107fc45baad66a5f223832afab49b306a0189b0
Sep 17 03:20:38.587127 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:38.586998    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b24v9"]
Sep 17 03:20:38.588055 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:38.587898    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9cp4t"]
Sep 17 03:20:38.589894 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:20:38.589851    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed49a17_07c3_4f80_8a00_7b5f21fbb61c.slice/crio-2476ed0179bce5b4ab935e29e9019dea56bf20614de5f57d228c09f4125dcaf3 WatchSource:0}: Error finding container 2476ed0179bce5b4ab935e29e9019dea56bf20614de5f57d228c09f4125dcaf3: Status 404 returned error can't find the container with id 2476ed0179bce5b4ab935e29e9019dea56bf20614de5f57d228c09f4125dcaf3
Sep 17 03:20:38.590853 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:20:38.590826    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb21d9f86_427c_4e22_89dd_f30258785552.slice/crio-1b671982a47b7f9a427bb054e4aae09453a8a527eea808021d6704b5c2e77492 WatchSource:0}: Error finding container 1b671982a47b7f9a427bb054e4aae09453a8a527eea808021d6704b5c2e77492: Status 404 returned error can't find the container with id 1b671982a47b7f9a427bb054e4aae09453a8a527eea808021d6704b5c2e77492
Sep 17 03:20:38.778030 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:38.777988    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs" event={"ID":"39eef97b-24d1-41dd-b78e-d0efe143494f","Type":"ContainerStarted","Data":"5514d81f1e33ab49a8a28fa23c47b2b51b38462ac4cf8b975fc2f506a9b82e1c"}
Sep 17 03:20:38.778224 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:38.778047    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs" event={"ID":"39eef97b-24d1-41dd-b78e-d0efe143494f","Type":"ContainerStarted","Data":"3eea8d71ba0a21209dbc231a2809837d5308049774498126b7711145ae06ff67"}
Sep 17 03:20:38.778224 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:38.778064    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs" event={"ID":"39eef97b-24d1-41dd-b78e-d0efe143494f","Type":"ContainerStarted","Data":"196469d7e196ad008e30aadcefb726468828a55aa36a15ca4d59a2dfcc6c5bb7"}
Sep 17 03:20:38.779236 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:38.779217    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9cp4t" event={"ID":"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c","Type":"ContainerStarted","Data":"2476ed0179bce5b4ab935e29e9019dea56bf20614de5f57d228c09f4125dcaf3"}
Sep 17 03:20:38.780446 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:38.780413    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nwswx" event={"ID":"6e66fbb6-8a12-4388-9b2b-4f4d256673cc","Type":"ContainerStarted","Data":"d48f8544343f9a483ce236f4c107fc45baad66a5f223832afab49b306a0189b0"}
Sep 17 03:20:38.781753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:38.781732    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h" event={"ID":"dfa820e0-5da0-43c4-86b6-9a80c393fb7a","Type":"ContainerStarted","Data":"a5c5a54f758c2eee70b235c22e412385ddf727b9bcda02dd9a269569e0bb0f53"}
Sep 17 03:20:38.782872 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:38.782846    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b24v9" event={"ID":"b21d9f86-427c-4e22-89dd-f30258785552","Type":"ContainerStarted","Data":"1b671982a47b7f9a427bb054e4aae09453a8a527eea808021d6704b5c2e77492"}
Sep 17 03:20:38.799041 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:38.798997    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-856444975-plsrs" podStartSLOduration=1.966873358 podStartE2EDuration="5.798981458s" totalImagesPullingTime="3.8321081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:20:38.797798247 +0000 UTC m=+67.142524331" watchObservedRunningTime="2026-09-17 03:20:38.798981458 +0000 UTC m=+67.143707589"
Sep 17 03:20:38.809804 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:38.809685    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-86d5f77c96-4mm2h" podStartSLOduration=1.9829761700000001 podStartE2EDuration="5.809664134s" totalImagesPullingTime="3.826687964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:20:38.809589899 +0000 UTC m=+67.154315979" watchObservedRunningTime="2026-09-17 03:20:38.809664134 +0000 UTC m=+67.154390214"
Sep 17 03:20:39.593017 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:39.592933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:39.689443 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:39.689389    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:39.785895 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:39.785831    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:40.090672 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:40.090590    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:40.793765 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:40.793723    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b24v9" event={"ID":"b21d9f86-427c-4e22-89dd-f30258785552","Type":"ContainerStarted","Data":"99da19b5ab624eeac2ce8e02f8bebfaea139caf66ef93d318acca4f0b051a896"}
Sep 17 03:20:40.794214 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:40.793789    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b24v9" event={"ID":"b21d9f86-427c-4e22-89dd-f30258785552","Type":"ContainerStarted","Data":"fb660d0c0210caf9329f9fd9f6b286d47cb70fa955d78588637d7e2c3eb57e45"}
Sep 17 03:20:40.808320 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:40.808012    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b24v9" podStartSLOduration=67.506772465 podStartE2EDuration="1m8.807991993s" totalImagesPullingTime="1.301219528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:20:40.806996023 +0000 UTC m=+69.151722107" watchObservedRunningTime="2026-09-17 03:20:40.807991993 +0000 UTC m=+69.152718082"
Sep 17 03:20:40.882607 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:40.882552    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:40.980377 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:40.980337    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:42.803699 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:42.803647    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:42.904687 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:42.904656    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:43.056492 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:43.056387    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:43.807070 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:43.807035    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9cp4t" event={"ID":"2ed49a17-07c3-4f80-8a00-7b5f21fbb61c","Type":"ContainerStarted","Data":"2f3fa01398d7976e07c89f561409dff28f615d214fa9cf58c34ab4d09a34cfe4"}
Sep 17 03:20:43.808552 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:43.808519    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nwswx" event={"ID":"6e66fbb6-8a12-4388-9b2b-4f4d256673cc","Type":"ContainerStarted","Data":"fbd2bfe48f676cf210e607f25b2ce5c6f24f3ee978c968d1f84e55b88fe57505"}
Sep 17 03:20:43.808679 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:43.808660    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:20:43.821249 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:43.821199    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9cp4t" podStartSLOduration=66.908021839 podStartE2EDuration="1m11.821181769s" totalImagesPullingTime="4.91315993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:20:43.820607297 +0000 UTC m=+72.165333380" watchObservedRunningTime="2026-09-17 03:20:43.821181769 +0000 UTC m=+72.165907909"
Sep 17 03:20:43.834216 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:43.834163    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nwswx" podStartSLOduration=66.919550515 podStartE2EDuration="1m11.834149142s" totalImagesPullingTime="4.914598627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:20:43.833333405 +0000 UTC m=+72.178059484" watchObservedRunningTime="2026-09-17 03:20:43.834149142 +0000 UTC m=+72.178875221"
Sep 17 03:20:43.840863 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:43.840813    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:43.915105 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:43.915068    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:45.163124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:45.163077    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:46.602630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:46.602550    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:46.659612 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:46.659583    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:46.925741 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:46.925636    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:47.161747 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.161707    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:47.204897 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.204801    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mh687"]
Sep 17 03:20:47.207511 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.207487    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.210169 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.210148    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 03:20:47.210282 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.210202    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c2z72\""
Sep 17 03:20:47.211319 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.211298    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 03:20:47.211471 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.211457    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 03:20:47.211526 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.211488    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 03:20:47.211848 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.211832    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 03:20:47.212010 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.211992    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 03:20:47.282711 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.282663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:47.302745 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.302708    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-wtmp\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.302923 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.302759    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-textfile\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.302923 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.302800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/672c0fb5-6604-47b5-b727-dd595e599157-sys\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.302923 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.302833    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4jqm\" (UniqueName: \"kubernetes.io/projected/672c0fb5-6604-47b5-b727-dd595e599157-kube-api-access-c4jqm\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.302923 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.302859    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/672c0fb5-6604-47b5-b727-dd595e599157-metrics-client-ca\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.302923 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.302890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/672c0fb5-6604-47b5-b727-dd595e599157-root\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.302923 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.302916    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.303137 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.302956    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-accelerators-collector-config\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.303137 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.303009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-tls\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.383881 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.383836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:47.403523 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.403483    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.403523 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.403520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-accelerators-collector-config\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.403773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.403540    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-tls\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.403773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.403560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-wtmp\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.403773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.403588    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-textfile\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.403773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.403612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/672c0fb5-6604-47b5-b727-dd595e599157-sys\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.403773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.403648    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4jqm\" (UniqueName: \"kubernetes.io/projected/672c0fb5-6604-47b5-b727-dd595e599157-kube-api-access-c4jqm\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.403773 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:47.403663    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 17 03:20:47.403773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.403674    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/672c0fb5-6604-47b5-b727-dd595e599157-metrics-client-ca\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.403773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.403712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/672c0fb5-6604-47b5-b727-dd595e599157-root\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.403773 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:47.403740    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-tls podName:672c0fb5-6604-47b5-b727-dd595e599157 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:47.903717716 +0000 UTC m=+76.248443796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-tls") pod "node-exporter-mh687" (UID: "672c0fb5-6604-47b5-b727-dd595e599157") : secret "node-exporter-tls" not found
Sep 17 03:20:47.403773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.403749    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/672c0fb5-6604-47b5-b727-dd595e599157-root\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.403773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.403735    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/672c0fb5-6604-47b5-b727-dd595e599157-sys\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.404221 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.403735    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-wtmp\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.404221 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.404017    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-textfile\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.404308 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.404247    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-accelerators-collector-config\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.404308 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.404254    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/672c0fb5-6604-47b5-b727-dd595e599157-metrics-client-ca\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.406061 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.406036    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.411785 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.411760    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4jqm\" (UniqueName: \"kubernetes.io/projected/672c0fb5-6604-47b5-b727-dd595e599157-kube-api-access-c4jqm\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.486857 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.486777    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:47.580619 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.580587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:47.681790 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.681750    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:47.782607 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.782511    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:47.907690 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:47.907651    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-tls\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:47.907858 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:47.907800    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 17 03:20:47.907898 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:20:47.907884    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-tls podName:672c0fb5-6604-47b5-b727-dd595e599157 nodeName:}" failed. No retries permitted until 2026-09-17 03:20:48.90786877 +0000 UTC m=+77.252594827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-tls") pod "node-exporter-mh687" (UID: "672c0fb5-6604-47b5-b727-dd595e599157") : secret "node-exporter-tls" not found
Sep 17 03:20:48.158146 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:48.158110    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:48.281101 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:48.281068    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:48.580657 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:48.580625    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:48.680539 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:48.680500    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:48.914656 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:48.914568    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-tls\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:48.916858 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:48.916824    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/672c0fb5-6604-47b5-b727-dd595e599157-node-exporter-tls\") pod \"node-exporter-mh687\" (UID: \"672c0fb5-6604-47b5-b727-dd595e599157\") " pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:49.016260 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:49.016199    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mh687"
Sep 17 03:20:49.026516 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:20:49.026489    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod672c0fb5_6604_47b5_b727_dd595e599157.slice/crio-beb448207362208ed28e0a7742a2af038293e5fbf825ed75dc4267362b9d1636 WatchSource:0}: Error finding container beb448207362208ed28e0a7742a2af038293e5fbf825ed75dc4267362b9d1636: Status 404 returned error can't find the container with id beb448207362208ed28e0a7742a2af038293e5fbf825ed75dc4267362b9d1636
Sep 17 03:20:49.829478 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:49.829438    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mh687" event={"ID":"672c0fb5-6604-47b5-b727-dd595e599157","Type":"ContainerStarted","Data":"beb448207362208ed28e0a7742a2af038293e5fbf825ed75dc4267362b9d1636"}
Sep 17 03:20:50.122550 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:50.122522    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:50.206075 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:50.206041    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:50.669974 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:50.669945    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:50.834012 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:50.833980    2576 generic.go:356] "Generic (PLEG): container finished" podID="672c0fb5-6604-47b5-b727-dd595e599157" containerID="32b7c2d8e32bca07bdc5cc50405bd346cf7f7ba71b5ac586a40fc135c43496fa" exitCode=0
Sep 17 03:20:50.834199 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:50.834021    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mh687" event={"ID":"672c0fb5-6604-47b5-b727-dd595e599157","Type":"ContainerDied","Data":"32b7c2d8e32bca07bdc5cc50405bd346cf7f7ba71b5ac586a40fc135c43496fa"}
Sep 17 03:20:50.944174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:50.944147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:51.840213 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:51.840176    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mh687" event={"ID":"672c0fb5-6604-47b5-b727-dd595e599157","Type":"ContainerStarted","Data":"642d751ce1efb48a4167528290f565c4bc70d60b836317182908a6536b68fc19"}
Sep 17 03:20:51.840628 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:51.840235    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mh687" event={"ID":"672c0fb5-6604-47b5-b727-dd595e599157","Type":"ContainerStarted","Data":"f20ae4a71f41b7bfc422f0c1cc1a087a115e7cdd99574aff28aefb1428318e00"}
Sep 17 03:20:51.857228 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:51.857169    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mh687" podStartSLOduration=3.843463764 podStartE2EDuration="4.857149575s" totalImagesPullingTime="945.302177ms" totalInitContainerRuntime="68.383634ms" isStatefulPod=true podCreationTimestamp="2026-09-17 03:20:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:20:51.856196452 +0000 UTC m=+80.200922531" watchObservedRunningTime="2026-09-17 03:20:51.857149575 +0000 UTC m=+80.201875653"
Sep 17 03:20:51.962410 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:51.962360    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:52.313899 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:52.313865    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:53.501694 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:53.501656    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:53.747229 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:53.747197    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:53.897761 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:53.897721    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:55.735858 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:55.735811    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:55.837019 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:55.836990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:56.570416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:56.570385    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:56.671327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:56.671298    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:58.678665 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:58.678622    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:20:58.746006 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:20:58.745961    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:00.788067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:00.788035    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:00.886559 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:00.886529    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:00.987185 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:00.987156    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:01.087893 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:01.087867    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:01.257321 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:01.257292    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:01.360022 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:01.359952    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:01.481780 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:01.481750    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:01.548900 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:01.548868    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:02.471523 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:02.471491    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:02.861311 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:02.861281    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:03.122081 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:03.121995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:03.685378 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:03.685343    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:03.787262 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:03.787231    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:06.968917 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:06.968881    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:07.025224 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:07.025194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:07.076446 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:07.076401    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:08.136423 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:08.136392    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:08.191006 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:08.190979    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:09.481666 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:09.481630    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:09.543725 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:09.543696    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:11.079452 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.079405    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78c49b459c-n5xh5"]
Sep 17 03:21:11.084606 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.084582    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.086779 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.086755    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 03:21:11.087406 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.087373    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 03:21:11.087406 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.087395    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 03:21:11.087601 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.087421    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 03:21:11.087601 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.087401    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5fg2h\""
Sep 17 03:21:11.087601 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.087377    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 03:21:11.087601 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.087443    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 03:21:11.087601 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.087375    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 03:21:11.096609 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.096226    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c49b459c-n5xh5"]
Sep 17 03:21:11.164343 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.164300    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-service-ca\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.164343 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.164346    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-console-config\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.164615 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.164377    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df6b1a7b-627f-499b-997f-abea51888f6a-console-serving-cert\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.164615 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.164401    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r68w\" (UniqueName: \"kubernetes.io/projected/df6b1a7b-627f-499b-997f-abea51888f6a-kube-api-access-9r68w\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.164615 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.164457    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df6b1a7b-627f-499b-997f-abea51888f6a-console-oauth-config\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.164615 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.164491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-oauth-serving-cert\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.265809 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.265769    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-console-config\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.265809 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.265815    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df6b1a7b-627f-499b-997f-abea51888f6a-console-serving-cert\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.266028 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.265833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r68w\" (UniqueName: \"kubernetes.io/projected/df6b1a7b-627f-499b-997f-abea51888f6a-kube-api-access-9r68w\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.266028 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.265863    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df6b1a7b-627f-499b-997f-abea51888f6a-console-oauth-config\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.266028 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.265888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-oauth-serving-cert\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.266028 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.265927    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-service-ca\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.266637 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.266611    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-service-ca\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.266637 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.266621    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-console-config\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.266805 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.266621    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-oauth-serving-cert\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.268287 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.268257    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df6b1a7b-627f-499b-997f-abea51888f6a-console-serving-cert\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.268382 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.268258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df6b1a7b-627f-499b-997f-abea51888f6a-console-oauth-config\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.276737 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.276719    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r68w\" (UniqueName: \"kubernetes.io/projected/df6b1a7b-627f-499b-997f-abea51888f6a-kube-api-access-9r68w\") pod \"console-78c49b459c-n5xh5\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") " pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.398657 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.398558    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:11.665351 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.665327    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c49b459c-n5xh5"]
Sep 17 03:21:11.667854 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:21:11.667823    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf6b1a7b_627f_499b_997f_abea51888f6a.slice/crio-ba6b7c115dbcb1bea415cd2332618e10d3ec44ac38e55e1ecdfb5075f2dfc1cd WatchSource:0}: Error finding container ba6b7c115dbcb1bea415cd2332618e10d3ec44ac38e55e1ecdfb5075f2dfc1cd: Status 404 returned error can't find the container with id ba6b7c115dbcb1bea415cd2332618e10d3ec44ac38e55e1ecdfb5075f2dfc1cd
Sep 17 03:21:11.905008 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:11.904968    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c49b459c-n5xh5" event={"ID":"df6b1a7b-627f-499b-997f-abea51888f6a","Type":"ContainerStarted","Data":"ba6b7c115dbcb1bea415cd2332618e10d3ec44ac38e55e1ecdfb5075f2dfc1cd"}
Sep 17 03:21:14.483268 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:14.483211    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:14.582949 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:14.582920    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:14.814252 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:14.814218    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nwswx"
Sep 17 03:21:14.918265 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:14.918229    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c49b459c-n5xh5" event={"ID":"df6b1a7b-627f-499b-997f-abea51888f6a","Type":"ContainerStarted","Data":"782ee259cca0bb760e28364eed695fd7a87b3fdf8dae5933378f2382f3de6e0b"}
Sep 17 03:21:14.933138 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:14.933092    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78c49b459c-n5xh5" podStartSLOduration=1.048831368 podStartE2EDuration="3.933077247s" totalImagesPullingTime="2.884245879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:21:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:21:14.932714032 +0000 UTC m=+103.277440112" watchObservedRunningTime="2026-09-17 03:21:14.933077247 +0000 UTC m=+103.277803328"
Sep 17 03:21:17.915000 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:17.914958    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:17.963623 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:17.963592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:21.399405 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:21.399360    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:21.399815 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:21.399466    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:21.404356 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:21.404331    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:21.944970 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:21.944941    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:21.963294 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:21.963265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:22.061282 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:22.061248    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:22.609080 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:22.609048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:22.712260 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:22.712230    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:22.810626 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:22.810592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:23.695472 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:23.695422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:23.796131 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:23.796098    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:26.690068 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:26.690034    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:26.743149 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:26.743115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:29.788200 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:29.788166    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78c49b459c-n5xh5"]
Sep 17 03:21:29.893905 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:29.893873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:29.948655 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:29.948623    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:30.005209 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:30.005172    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:36.962581 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:36.962549    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:37.061515 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:37.061477    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:37.538444 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:37.538407    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-44gmz_fcc07e33-2671-4c55-9211-9714c8799a6e/serve-healthcheck-canary/0.log"
Sep 17 03:21:44.066975 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:44.066941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:44.168204 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:44.168172    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:44.227532 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:44.227503    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:21:54.807354 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:54.807289    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78c49b459c-n5xh5" podUID="df6b1a7b-627f-499b-997f-abea51888f6a" containerName="console" containerID="cri-o://782ee259cca0bb760e28364eed695fd7a87b3fdf8dae5933378f2382f3de6e0b" gracePeriod=15
Sep 17 03:21:55.050596 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.050568    2576 generic.go:356] "Generic (PLEG): container finished" podID="df6b1a7b-627f-499b-997f-abea51888f6a" containerID="782ee259cca0bb760e28364eed695fd7a87b3fdf8dae5933378f2382f3de6e0b" exitCode=0
Sep 17 03:21:55.050732 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.050644    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c49b459c-n5xh5" event={"ID":"df6b1a7b-627f-499b-997f-abea51888f6a","Type":"ContainerDied","Data":"782ee259cca0bb760e28364eed695fd7a87b3fdf8dae5933378f2382f3de6e0b"}
Sep 17 03:21:55.050732 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.050701    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c49b459c-n5xh5" event={"ID":"df6b1a7b-627f-499b-997f-abea51888f6a","Type":"ContainerDied","Data":"ba6b7c115dbcb1bea415cd2332618e10d3ec44ac38e55e1ecdfb5075f2dfc1cd"}
Sep 17 03:21:55.050732 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.050720    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba6b7c115dbcb1bea415cd2332618e10d3ec44ac38e55e1ecdfb5075f2dfc1cd"
Sep 17 03:21:55.051576 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.051562    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:55.176740 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.176634    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-service-ca\") pod \"df6b1a7b-627f-499b-997f-abea51888f6a\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") "
Sep 17 03:21:55.176740 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.176693    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df6b1a7b-627f-499b-997f-abea51888f6a-console-oauth-config\") pod \"df6b1a7b-627f-499b-997f-abea51888f6a\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") "
Sep 17 03:21:55.176740 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.176719    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-console-config\") pod \"df6b1a7b-627f-499b-997f-abea51888f6a\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") "
Sep 17 03:21:55.176740 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.176739    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df6b1a7b-627f-499b-997f-abea51888f6a-console-serving-cert\") pod \"df6b1a7b-627f-499b-997f-abea51888f6a\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") "
Sep 17 03:21:55.177056 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.176769    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-oauth-serving-cert\") pod \"df6b1a7b-627f-499b-997f-abea51888f6a\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") "
Sep 17 03:21:55.177056 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.176840    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r68w\" (UniqueName: \"kubernetes.io/projected/df6b1a7b-627f-499b-997f-abea51888f6a-kube-api-access-9r68w\") pod \"df6b1a7b-627f-499b-997f-abea51888f6a\" (UID: \"df6b1a7b-627f-499b-997f-abea51888f6a\") "
Sep 17 03:21:55.177190 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.177164    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-console-config" (OuterVolumeSpecName: "console-config") pod "df6b1a7b-627f-499b-997f-abea51888f6a" (UID: "df6b1a7b-627f-499b-997f-abea51888f6a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:21:55.177251 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.177075    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-service-ca" (OuterVolumeSpecName: "service-ca") pod "df6b1a7b-627f-499b-997f-abea51888f6a" (UID: "df6b1a7b-627f-499b-997f-abea51888f6a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:21:55.177302 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.177260    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "df6b1a7b-627f-499b-997f-abea51888f6a" (UID: "df6b1a7b-627f-499b-997f-abea51888f6a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:21:55.179087 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.179061    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df6b1a7b-627f-499b-997f-abea51888f6a-kube-api-access-9r68w" (OuterVolumeSpecName: "kube-api-access-9r68w") pod "df6b1a7b-627f-499b-997f-abea51888f6a" (UID: "df6b1a7b-627f-499b-997f-abea51888f6a"). InnerVolumeSpecName "kube-api-access-9r68w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:21:55.179168 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.179081    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df6b1a7b-627f-499b-997f-abea51888f6a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "df6b1a7b-627f-499b-997f-abea51888f6a" (UID: "df6b1a7b-627f-499b-997f-abea51888f6a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:21:55.179168 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.179066    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df6b1a7b-627f-499b-997f-abea51888f6a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "df6b1a7b-627f-499b-997f-abea51888f6a" (UID: "df6b1a7b-627f-499b-997f-abea51888f6a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:21:55.277376 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.277339    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/df6b1a7b-627f-499b-997f-abea51888f6a-console-oauth-config\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:21:55.277376 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.277366    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-console-config\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:21:55.277376 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.277375    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/df6b1a7b-627f-499b-997f-abea51888f6a-console-serving-cert\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:21:55.277655 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.277421    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-oauth-serving-cert\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:21:55.277655 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.277446    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r68w\" (UniqueName: \"kubernetes.io/projected/df6b1a7b-627f-499b-997f-abea51888f6a-kube-api-access-9r68w\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:21:55.277655 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:55.277461    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/df6b1a7b-627f-499b-997f-abea51888f6a-service-ca\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:21:56.053874 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:56.053841    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c49b459c-n5xh5"
Sep 17 03:21:56.071972 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:56.071948    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78c49b459c-n5xh5"]
Sep 17 03:21:56.075174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:56.075154    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78c49b459c-n5xh5"]
Sep 17 03:21:56.292491 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:21:56.292462    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df6b1a7b-627f-499b-997f-abea51888f6a" path="/var/lib/kubelet/pods/df6b1a7b-627f-499b-997f-abea51888f6a/volumes"
Sep 17 03:22:03.300028 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:03.299985    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:22:03.400533 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:03.400485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:22:15.276148 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:15.276112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:22:15.377736 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:15.377695    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:22:15.771758 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:15.771720    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:22:15.824775 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:15.824741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:22:15.918993 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:15.918961    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:22:16.519138 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:16.519107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:22:16.620112 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:16.620082    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:22:21.967867 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:21.967826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:22:22.070927 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:22.070892    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:22:27.165616 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:27.165533    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:22:27.221251 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:27.221217    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:22:27.274395 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:22:27.274361    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:07.584158 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:07.584126    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:07.648287 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:07.648255    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:17.288526 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:17.288478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:17.385875 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:17.385843    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:17.944384 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:17.944351    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"]
Sep 17 03:23:17.944714 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:17.944697    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df6b1a7b-627f-499b-997f-abea51888f6a" containerName="console"
Sep 17 03:23:17.944770 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:17.944717    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="df6b1a7b-627f-499b-997f-abea51888f6a" containerName="console"
Sep 17 03:23:17.944810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:17.944777    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="df6b1a7b-627f-499b-997f-abea51888f6a" containerName="console"
Sep 17 03:23:17.947681 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:17.947665    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"
Sep 17 03:23:17.949723 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:17.949705    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 03:23:17.949822 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:17.949784    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4svbt\""
Sep 17 03:23:17.950306 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:17.950290    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 03:23:17.956116 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:17.956091    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"]
Sep 17 03:23:18.004979 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:18.004938    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/582e23aa-93ae-488a-b78e-caf03fe27fe9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm\" (UID: \"582e23aa-93ae-488a-b78e-caf03fe27fe9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"
Sep 17 03:23:18.004979 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:18.004976    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/582e23aa-93ae-488a-b78e-caf03fe27fe9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm\" (UID: \"582e23aa-93ae-488a-b78e-caf03fe27fe9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"
Sep 17 03:23:18.005204 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:18.004996    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plzcn\" (UniqueName: \"kubernetes.io/projected/582e23aa-93ae-488a-b78e-caf03fe27fe9-kube-api-access-plzcn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm\" (UID: \"582e23aa-93ae-488a-b78e-caf03fe27fe9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"
Sep 17 03:23:18.105803 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:18.105765    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plzcn\" (UniqueName: \"kubernetes.io/projected/582e23aa-93ae-488a-b78e-caf03fe27fe9-kube-api-access-plzcn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm\" (UID: \"582e23aa-93ae-488a-b78e-caf03fe27fe9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"
Sep 17 03:23:18.105959 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:18.105867    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/582e23aa-93ae-488a-b78e-caf03fe27fe9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm\" (UID: \"582e23aa-93ae-488a-b78e-caf03fe27fe9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"
Sep 17 03:23:18.105959 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:18.105889    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/582e23aa-93ae-488a-b78e-caf03fe27fe9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm\" (UID: \"582e23aa-93ae-488a-b78e-caf03fe27fe9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"
Sep 17 03:23:18.106331 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:18.106310    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/582e23aa-93ae-488a-b78e-caf03fe27fe9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm\" (UID: \"582e23aa-93ae-488a-b78e-caf03fe27fe9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"
Sep 17 03:23:18.106367 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:18.106324    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/582e23aa-93ae-488a-b78e-caf03fe27fe9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm\" (UID: \"582e23aa-93ae-488a-b78e-caf03fe27fe9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"
Sep 17 03:23:18.115171 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:18.115132    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plzcn\" (UniqueName: \"kubernetes.io/projected/582e23aa-93ae-488a-b78e-caf03fe27fe9-kube-api-access-plzcn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm\" (UID: \"582e23aa-93ae-488a-b78e-caf03fe27fe9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"
Sep 17 03:23:18.257088 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:18.257003    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"
Sep 17 03:23:18.526791 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:18.526716    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"]
Sep 17 03:23:18.529659 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:23:18.529620    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod582e23aa_93ae_488a_b78e_caf03fe27fe9.slice/crio-f1f3bf555b984fe580d063096cd6570a985277513291b7e78a01ab5334ad6d79 WatchSource:0}: Error finding container f1f3bf555b984fe580d063096cd6570a985277513291b7e78a01ab5334ad6d79: Status 404 returned error can't find the container with id f1f3bf555b984fe580d063096cd6570a985277513291b7e78a01ab5334ad6d79
Sep 17 03:23:19.338458 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:19.338405    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm" event={"ID":"582e23aa-93ae-488a-b78e-caf03fe27fe9","Type":"ContainerStarted","Data":"f1f3bf555b984fe580d063096cd6570a985277513291b7e78a01ab5334ad6d79"}
Sep 17 03:23:25.362861 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:25.362824    2576 generic.go:356] "Generic (PLEG): container finished" podID="582e23aa-93ae-488a-b78e-caf03fe27fe9" containerID="a7f6a977fde690972fc72270c7de0ce424625682c8dbfc19001090866e8a1900" exitCode=0
Sep 17 03:23:25.363258 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:25.362912    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm" event={"ID":"582e23aa-93ae-488a-b78e-caf03fe27fe9","Type":"ContainerDied","Data":"a7f6a977fde690972fc72270c7de0ce424625682c8dbfc19001090866e8a1900"}
Sep 17 03:23:28.377313 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:28.377273    2576 generic.go:356] "Generic (PLEG): container finished" podID="582e23aa-93ae-488a-b78e-caf03fe27fe9" containerID="576848d33376c18d678ee3593e9478ea6914db4987f0b8c498d462469477ec69" exitCode=0
Sep 17 03:23:28.377739 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:28.377353    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm" event={"ID":"582e23aa-93ae-488a-b78e-caf03fe27fe9","Type":"ContainerDied","Data":"576848d33376c18d678ee3593e9478ea6914db4987f0b8c498d462469477ec69"}
Sep 17 03:23:32.219596 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:32.219563    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:32.270863 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:32.270813    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:35.405289 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:35.405254    2576 generic.go:356] "Generic (PLEG): container finished" podID="582e23aa-93ae-488a-b78e-caf03fe27fe9" containerID="1cbe5fadd87b4cefcd5e3532b676a3c73ba293365bd891bca5cafe6d4be60531" exitCode=0
Sep 17 03:23:35.405686 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:35.405337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm" event={"ID":"582e23aa-93ae-488a-b78e-caf03fe27fe9","Type":"ContainerDied","Data":"1cbe5fadd87b4cefcd5e3532b676a3c73ba293365bd891bca5cafe6d4be60531"}
Sep 17 03:23:36.529426 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:36.529402    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"
Sep 17 03:23:36.635032 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:36.634997    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/582e23aa-93ae-488a-b78e-caf03fe27fe9-bundle\") pod \"582e23aa-93ae-488a-b78e-caf03fe27fe9\" (UID: \"582e23aa-93ae-488a-b78e-caf03fe27fe9\") "
Sep 17 03:23:36.635032 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:36.635051    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plzcn\" (UniqueName: \"kubernetes.io/projected/582e23aa-93ae-488a-b78e-caf03fe27fe9-kube-api-access-plzcn\") pod \"582e23aa-93ae-488a-b78e-caf03fe27fe9\" (UID: \"582e23aa-93ae-488a-b78e-caf03fe27fe9\") "
Sep 17 03:23:36.635246 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:36.635097    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/582e23aa-93ae-488a-b78e-caf03fe27fe9-util\") pod \"582e23aa-93ae-488a-b78e-caf03fe27fe9\" (UID: \"582e23aa-93ae-488a-b78e-caf03fe27fe9\") "
Sep 17 03:23:36.635652 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:36.635622    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582e23aa-93ae-488a-b78e-caf03fe27fe9-bundle" (OuterVolumeSpecName: "bundle") pod "582e23aa-93ae-488a-b78e-caf03fe27fe9" (UID: "582e23aa-93ae-488a-b78e-caf03fe27fe9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:23:36.637282 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:36.637250    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/582e23aa-93ae-488a-b78e-caf03fe27fe9-kube-api-access-plzcn" (OuterVolumeSpecName: "kube-api-access-plzcn") pod "582e23aa-93ae-488a-b78e-caf03fe27fe9" (UID: "582e23aa-93ae-488a-b78e-caf03fe27fe9"). InnerVolumeSpecName "kube-api-access-plzcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:23:36.639325 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:36.639304    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582e23aa-93ae-488a-b78e-caf03fe27fe9-util" (OuterVolumeSpecName: "util") pod "582e23aa-93ae-488a-b78e-caf03fe27fe9" (UID: "582e23aa-93ae-488a-b78e-caf03fe27fe9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:23:36.735536 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:36.735457    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/582e23aa-93ae-488a-b78e-caf03fe27fe9-util\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:23:36.735536 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:36.735483    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/582e23aa-93ae-488a-b78e-caf03fe27fe9-bundle\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:23:36.735536 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:36.735493    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plzcn\" (UniqueName: \"kubernetes.io/projected/582e23aa-93ae-488a-b78e-caf03fe27fe9-kube-api-access-plzcn\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:23:37.414998 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:37.414959    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm" event={"ID":"582e23aa-93ae-488a-b78e-caf03fe27fe9","Type":"ContainerDied","Data":"f1f3bf555b984fe580d063096cd6570a985277513291b7e78a01ab5334ad6d79"}
Sep 17 03:23:37.414998 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:37.414999    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vbbsm"
Sep 17 03:23:37.415204 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:37.415019    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1f3bf555b984fe580d063096cd6570a985277513291b7e78a01ab5334ad6d79"
Sep 17 03:23:38.908817 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:38.908787    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:39.719733 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:39.719692    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:40.880991 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.880961    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg"]
Sep 17 03:23:40.881358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.881194    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="582e23aa-93ae-488a-b78e-caf03fe27fe9" containerName="pull"
Sep 17 03:23:40.881358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.881204    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="582e23aa-93ae-488a-b78e-caf03fe27fe9" containerName="pull"
Sep 17 03:23:40.881358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.881211    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="582e23aa-93ae-488a-b78e-caf03fe27fe9" containerName="extract"
Sep 17 03:23:40.881358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.881218    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="582e23aa-93ae-488a-b78e-caf03fe27fe9" containerName="extract"
Sep 17 03:23:40.881358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.881228    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="582e23aa-93ae-488a-b78e-caf03fe27fe9" containerName="util"
Sep 17 03:23:40.881358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.881233    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="582e23aa-93ae-488a-b78e-caf03fe27fe9" containerName="util"
Sep 17 03:23:40.881358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.881269    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="582e23aa-93ae-488a-b78e-caf03fe27fe9" containerName="extract"
Sep 17 03:23:40.914213 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.914177    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg"]
Sep 17 03:23:40.914369 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.914274    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg"
Sep 17 03:23:40.916414 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.916387    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 17 03:23:40.916569 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.916512    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 17 03:23:40.916630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.916580    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-htvfh\""
Sep 17 03:23:40.963581 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.963548    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6rgf\" (UniqueName: \"kubernetes.io/projected/6c53a7b9-0ab2-456f-a34b-72bce32a87c9-kube-api-access-m6rgf\") pod \"cert-manager-operator-controller-manager-55c444bf86-p94pg\" (UID: \"6c53a7b9-0ab2-456f-a34b-72bce32a87c9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg"
Sep 17 03:23:40.963747 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:40.963600    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c53a7b9-0ab2-456f-a34b-72bce32a87c9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-p94pg\" (UID: \"6c53a7b9-0ab2-456f-a34b-72bce32a87c9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg"
Sep 17 03:23:41.064072 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:41.064030    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c53a7b9-0ab2-456f-a34b-72bce32a87c9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-p94pg\" (UID: \"6c53a7b9-0ab2-456f-a34b-72bce32a87c9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg"
Sep 17 03:23:41.064242 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:41.064114    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6rgf\" (UniqueName: \"kubernetes.io/projected/6c53a7b9-0ab2-456f-a34b-72bce32a87c9-kube-api-access-m6rgf\") pod \"cert-manager-operator-controller-manager-55c444bf86-p94pg\" (UID: \"6c53a7b9-0ab2-456f-a34b-72bce32a87c9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg"
Sep 17 03:23:41.064506 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:41.064486    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c53a7b9-0ab2-456f-a34b-72bce32a87c9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-p94pg\" (UID: \"6c53a7b9-0ab2-456f-a34b-72bce32a87c9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg"
Sep 17 03:23:41.076439 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:41.076403    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6rgf\" (UniqueName: \"kubernetes.io/projected/6c53a7b9-0ab2-456f-a34b-72bce32a87c9-kube-api-access-m6rgf\") pod \"cert-manager-operator-controller-manager-55c444bf86-p94pg\" (UID: \"6c53a7b9-0ab2-456f-a34b-72bce32a87c9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg"
Sep 17 03:23:41.223283 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:41.223202    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg"
Sep 17 03:23:41.525058 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:41.524974    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg"]
Sep 17 03:23:41.528279 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:23:41.528249    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c53a7b9_0ab2_456f_a34b_72bce32a87c9.slice/crio-533dfbf3b386a6a6dd3170d73bdb5c108f1f53f048f105adc2adad49f178eb77 WatchSource:0}: Error finding container 533dfbf3b386a6a6dd3170d73bdb5c108f1f53f048f105adc2adad49f178eb77: Status 404 returned error can't find the container with id 533dfbf3b386a6a6dd3170d73bdb5c108f1f53f048f105adc2adad49f178eb77
Sep 17 03:23:42.435155 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:42.435117    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg" event={"ID":"6c53a7b9-0ab2-456f-a34b-72bce32a87c9","Type":"ContainerStarted","Data":"533dfbf3b386a6a6dd3170d73bdb5c108f1f53f048f105adc2adad49f178eb77"}
Sep 17 03:23:42.516035 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:42.515996    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:42.565959 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:42.565908    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:44.443471 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:44.443418    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg" event={"ID":"6c53a7b9-0ab2-456f-a34b-72bce32a87c9","Type":"ContainerStarted","Data":"a5967e99642f57491755340d758e18297d7f021ba62d8b2508bbf5d9717ee0eb"}
Sep 17 03:23:44.460893 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:44.460843    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-p94pg" podStartSLOduration=2.208043023 podStartE2EDuration="4.460825355s" totalImagesPullingTime="2.252782332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:23:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:23:44.45956388 +0000 UTC m=+252.804289972" watchObservedRunningTime="2026-09-17 03:23:44.460825355 +0000 UTC m=+252.805551434"
Sep 17 03:23:44.751746 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:44.751660    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:44.863841 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:44.863811    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:44.958316 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:44.958285    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:45.360864 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.360824    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"]
Sep 17 03:23:45.363241 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.363215    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"
Sep 17 03:23:45.365272 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.365250    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 03:23:45.365395 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.365249    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4svbt\""
Sep 17 03:23:45.365756 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.365738    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 03:23:45.373873 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.373850    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"]
Sep 17 03:23:45.396096 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.396066    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp4pb\" (UniqueName: \"kubernetes.io/projected/1e61f51e-fe2c-48bd-a95f-98de1c583fea-kube-api-access-cp4pb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j\" (UID: \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"
Sep 17 03:23:45.396250 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.396102    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e61f51e-fe2c-48bd-a95f-98de1c583fea-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j\" (UID: \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"
Sep 17 03:23:45.396250 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.396125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e61f51e-fe2c-48bd-a95f-98de1c583fea-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j\" (UID: \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"
Sep 17 03:23:45.496734 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.496692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e61f51e-fe2c-48bd-a95f-98de1c583fea-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j\" (UID: \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"
Sep 17 03:23:45.497219 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.496744    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e61f51e-fe2c-48bd-a95f-98de1c583fea-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j\" (UID: \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"
Sep 17 03:23:45.497219 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.496819    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp4pb\" (UniqueName: \"kubernetes.io/projected/1e61f51e-fe2c-48bd-a95f-98de1c583fea-kube-api-access-cp4pb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j\" (UID: \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"
Sep 17 03:23:45.497219 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.497090    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e61f51e-fe2c-48bd-a95f-98de1c583fea-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j\" (UID: \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"
Sep 17 03:23:45.497219 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.497114    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e61f51e-fe2c-48bd-a95f-98de1c583fea-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j\" (UID: \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"
Sep 17 03:23:45.506302 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.506269    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp4pb\" (UniqueName: \"kubernetes.io/projected/1e61f51e-fe2c-48bd-a95f-98de1c583fea-kube-api-access-cp4pb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j\" (UID: \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"
Sep 17 03:23:45.672701 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.672601    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"
Sep 17 03:23:45.942071 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:45.941997    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"]
Sep 17 03:23:45.944905 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:23:45.944870    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e61f51e_fe2c_48bd_a95f_98de1c583fea.slice/crio-9d2182efb92b4effc0d759efc6f4ff1286dcf63d1ce514f8239eb37b64ec0cc1 WatchSource:0}: Error finding container 9d2182efb92b4effc0d759efc6f4ff1286dcf63d1ce514f8239eb37b64ec0cc1: Status 404 returned error can't find the container with id 9d2182efb92b4effc0d759efc6f4ff1286dcf63d1ce514f8239eb37b64ec0cc1
Sep 17 03:23:46.452014 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:46.451979    2576 generic.go:356] "Generic (PLEG): container finished" podID="1e61f51e-fe2c-48bd-a95f-98de1c583fea" containerID="00d6d845ed57a1fdc18fa3a30de7e7e56df562c2fefa768d8d26144aec1f9080" exitCode=0
Sep 17 03:23:46.452189 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:46.452034    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j" event={"ID":"1e61f51e-fe2c-48bd-a95f-98de1c583fea","Type":"ContainerDied","Data":"00d6d845ed57a1fdc18fa3a30de7e7e56df562c2fefa768d8d26144aec1f9080"}
Sep 17 03:23:46.452189 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:46.452079    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j" event={"ID":"1e61f51e-fe2c-48bd-a95f-98de1c583fea","Type":"ContainerStarted","Data":"9d2182efb92b4effc0d759efc6f4ff1286dcf63d1ce514f8239eb37b64ec0cc1"}
Sep 17 03:23:47.138041 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:47.137992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:47.337540 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:47.337505    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:49.467294 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:49.467243    2576 generic.go:356] "Generic (PLEG): container finished" podID="1e61f51e-fe2c-48bd-a95f-98de1c583fea" containerID="46d44a5adb76a9325c2e3c61d9277a5bb3bdd64d66a8ae86e6da996e3ef02623" exitCode=0
Sep 17 03:23:49.467681 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:49.467299    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j" event={"ID":"1e61f51e-fe2c-48bd-a95f-98de1c583fea","Type":"ContainerDied","Data":"46d44a5adb76a9325c2e3c61d9277a5bb3bdd64d66a8ae86e6da996e3ef02623"}
Sep 17 03:23:49.513037 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:49.513009    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:49.613610 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:49.613577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:50.473296 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:50.473265    2576 generic.go:356] "Generic (PLEG): container finished" podID="1e61f51e-fe2c-48bd-a95f-98de1c583fea" containerID="d3f73961fbd7790e30af6f2fe45fc3ef7209433bb838243a1875484b0a132b04" exitCode=0
Sep 17 03:23:50.473713 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:50.473329    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j" event={"ID":"1e61f51e-fe2c-48bd-a95f-98de1c583fea","Type":"ContainerDied","Data":"d3f73961fbd7790e30af6f2fe45fc3ef7209433bb838243a1875484b0a132b04"}
Sep 17 03:23:51.599777 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:51.599752    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"
Sep 17 03:23:51.644607 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:51.644574    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e61f51e-fe2c-48bd-a95f-98de1c583fea-util\") pod \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\" (UID: \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\") "
Sep 17 03:23:51.644790 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:51.644616    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e61f51e-fe2c-48bd-a95f-98de1c583fea-bundle\") pod \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\" (UID: \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\") "
Sep 17 03:23:51.644790 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:51.644649    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cp4pb\" (UniqueName: \"kubernetes.io/projected/1e61f51e-fe2c-48bd-a95f-98de1c583fea-kube-api-access-cp4pb\") pod \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\" (UID: \"1e61f51e-fe2c-48bd-a95f-98de1c583fea\") "
Sep 17 03:23:51.645072 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:51.645038    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e61f51e-fe2c-48bd-a95f-98de1c583fea-bundle" (OuterVolumeSpecName: "bundle") pod "1e61f51e-fe2c-48bd-a95f-98de1c583fea" (UID: "1e61f51e-fe2c-48bd-a95f-98de1c583fea"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:23:51.646782 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:51.646753    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e61f51e-fe2c-48bd-a95f-98de1c583fea-kube-api-access-cp4pb" (OuterVolumeSpecName: "kube-api-access-cp4pb") pod "1e61f51e-fe2c-48bd-a95f-98de1c583fea" (UID: "1e61f51e-fe2c-48bd-a95f-98de1c583fea"). InnerVolumeSpecName "kube-api-access-cp4pb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:23:51.649298 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:51.649269    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e61f51e-fe2c-48bd-a95f-98de1c583fea-util" (OuterVolumeSpecName: "util") pod "1e61f51e-fe2c-48bd-a95f-98de1c583fea" (UID: "1e61f51e-fe2c-48bd-a95f-98de1c583fea"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:23:51.745663 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:51.745569    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e61f51e-fe2c-48bd-a95f-98de1c583fea-util\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:23:51.745663 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:51.745598    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e61f51e-fe2c-48bd-a95f-98de1c583fea-bundle\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:23:51.745663 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:51.745608    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cp4pb\" (UniqueName: \"kubernetes.io/projected/1e61f51e-fe2c-48bd-a95f-98de1c583fea-kube-api-access-cp4pb\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:23:52.294078 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.294048    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-s22rg"]
Sep 17 03:23:52.294274 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.294263    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e61f51e-fe2c-48bd-a95f-98de1c583fea" containerName="pull"
Sep 17 03:23:52.294318 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.294276    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e61f51e-fe2c-48bd-a95f-98de1c583fea" containerName="pull"
Sep 17 03:23:52.294318 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.294289    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e61f51e-fe2c-48bd-a95f-98de1c583fea" containerName="util"
Sep 17 03:23:52.294318 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.294295    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e61f51e-fe2c-48bd-a95f-98de1c583fea" containerName="util"
Sep 17 03:23:52.294318 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.294301    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e61f51e-fe2c-48bd-a95f-98de1c583fea" containerName="extract"
Sep 17 03:23:52.294318 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.294306    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e61f51e-fe2c-48bd-a95f-98de1c583fea" containerName="extract"
Sep 17 03:23:52.294501 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.294351    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e61f51e-fe2c-48bd-a95f-98de1c583fea" containerName="extract"
Sep 17 03:23:52.296067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.296051    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-s22rg"
Sep 17 03:23:52.297856 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.297836    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 17 03:23:52.298283 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.298268    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 17 03:23:52.298383 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.298271    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-4gjlp\""
Sep 17 03:23:52.306824 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.306800    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-s22rg"]
Sep 17 03:23:52.349839 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.349796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjjhf\" (UniqueName: \"kubernetes.io/projected/06720a24-0dd8-4a24-b92a-079b7e411342-kube-api-access-vjjhf\") pod \"cert-manager-webhook-6486ffd997-s22rg\" (UID: \"06720a24-0dd8-4a24-b92a-079b7e411342\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s22rg"
Sep 17 03:23:52.350033 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.349865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06720a24-0dd8-4a24-b92a-079b7e411342-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-s22rg\" (UID: \"06720a24-0dd8-4a24-b92a-079b7e411342\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s22rg"
Sep 17 03:23:52.450183 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.450141    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06720a24-0dd8-4a24-b92a-079b7e411342-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-s22rg\" (UID: \"06720a24-0dd8-4a24-b92a-079b7e411342\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s22rg"
Sep 17 03:23:52.450355 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.450202    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjjhf\" (UniqueName: \"kubernetes.io/projected/06720a24-0dd8-4a24-b92a-079b7e411342-kube-api-access-vjjhf\") pod \"cert-manager-webhook-6486ffd997-s22rg\" (UID: \"06720a24-0dd8-4a24-b92a-079b7e411342\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s22rg"
Sep 17 03:23:52.457572 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.457537    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06720a24-0dd8-4a24-b92a-079b7e411342-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-s22rg\" (UID: \"06720a24-0dd8-4a24-b92a-079b7e411342\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s22rg"
Sep 17 03:23:52.457740 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.457703    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjjhf\" (UniqueName: \"kubernetes.io/projected/06720a24-0dd8-4a24-b92a-079b7e411342-kube-api-access-vjjhf\") pod \"cert-manager-webhook-6486ffd997-s22rg\" (UID: \"06720a24-0dd8-4a24-b92a-079b7e411342\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s22rg"
Sep 17 03:23:52.486389 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.486349    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j" event={"ID":"1e61f51e-fe2c-48bd-a95f-98de1c583fea","Type":"ContainerDied","Data":"9d2182efb92b4effc0d759efc6f4ff1286dcf63d1ce514f8239eb37b64ec0cc1"}
Sep 17 03:23:52.486389 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.486390    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvb57j"
Sep 17 03:23:52.486616 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.486448    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d2182efb92b4effc0d759efc6f4ff1286dcf63d1ce514f8239eb37b64ec0cc1"
Sep 17 03:23:52.606279 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.606232    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-s22rg"
Sep 17 03:23:52.873642 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:52.873618    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-s22rg"]
Sep 17 03:23:52.875699 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:23:52.875672    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06720a24_0dd8_4a24_b92a_079b7e411342.slice/crio-dad705dbf99ad90caaab53eeb09453cad7d50633dbe27a9483fe9664cc0c4e97 WatchSource:0}: Error finding container dad705dbf99ad90caaab53eeb09453cad7d50633dbe27a9483fe9664cc0c4e97: Status 404 returned error can't find the container with id dad705dbf99ad90caaab53eeb09453cad7d50633dbe27a9483fe9664cc0c4e97
Sep 17 03:23:53.492967 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:53.492885    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-s22rg" event={"ID":"06720a24-0dd8-4a24-b92a-079b7e411342","Type":"ContainerStarted","Data":"dad705dbf99ad90caaab53eeb09453cad7d50633dbe27a9483fe9664cc0c4e97"}
Sep 17 03:23:54.730222 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:54.730150    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:55.332248 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:55.332222    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:55.429368 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:55.429333    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:55.502527 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:55.502488    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-s22rg" event={"ID":"06720a24-0dd8-4a24-b92a-079b7e411342","Type":"ContainerStarted","Data":"f9394441c7cb3b73811464a47375716d05f26b3710715f524d222f751ef0fc20"}
Sep 17 03:23:55.502696 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:55.502648    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-s22rg"
Sep 17 03:23:55.520302 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:55.520254    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-s22rg" podStartSLOduration=1.096962874 podStartE2EDuration="3.520238344s" totalImagesPullingTime="2.42327547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:23:55.518697733 +0000 UTC m=+263.863423822" watchObservedRunningTime="2026-09-17 03:23:55.520238344 +0000 UTC m=+263.864964423"
Sep 17 03:23:55.538364 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:55.538340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:55.595943 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:55.595913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:57.419761 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.419724    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d"]
Sep 17 03:23:57.421958 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.421935    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d"
Sep 17 03:23:57.423807 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.423781    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 17 03:23:57.423922 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.423822    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 17 03:23:57.424381 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.424361    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-6qkqj\""
Sep 17 03:23:57.430463 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.430421    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d"]
Sep 17 03:23:57.490030 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.489991    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ae8e927-56ef-48da-951e-3f7e18968c68-tmp\") pod \"openshift-lws-operator-fd8ccff4c-s6t6d\" (UID: \"0ae8e927-56ef-48da-951e-3f7e18968c68\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d"
Sep 17 03:23:57.490210 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.490042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8rmw\" (UniqueName: \"kubernetes.io/projected/0ae8e927-56ef-48da-951e-3f7e18968c68-kube-api-access-v8rmw\") pod \"openshift-lws-operator-fd8ccff4c-s6t6d\" (UID: \"0ae8e927-56ef-48da-951e-3f7e18968c68\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d"
Sep 17 03:23:57.590741 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.590696    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ae8e927-56ef-48da-951e-3f7e18968c68-tmp\") pod \"openshift-lws-operator-fd8ccff4c-s6t6d\" (UID: \"0ae8e927-56ef-48da-951e-3f7e18968c68\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d"
Sep 17 03:23:57.590914 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.590753    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8rmw\" (UniqueName: \"kubernetes.io/projected/0ae8e927-56ef-48da-951e-3f7e18968c68-kube-api-access-v8rmw\") pod \"openshift-lws-operator-fd8ccff4c-s6t6d\" (UID: \"0ae8e927-56ef-48da-951e-3f7e18968c68\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d"
Sep 17 03:23:57.591108 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.591087    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ae8e927-56ef-48da-951e-3f7e18968c68-tmp\") pod \"openshift-lws-operator-fd8ccff4c-s6t6d\" (UID: \"0ae8e927-56ef-48da-951e-3f7e18968c68\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d"
Sep 17 03:23:57.599221 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.599186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8rmw\" (UniqueName: \"kubernetes.io/projected/0ae8e927-56ef-48da-951e-3f7e18968c68-kube-api-access-v8rmw\") pod \"openshift-lws-operator-fd8ccff4c-s6t6d\" (UID: \"0ae8e927-56ef-48da-951e-3f7e18968c68\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d"
Sep 17 03:23:57.731192 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.731088    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d"
Sep 17 03:23:57.997975 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:57.997865    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d"]
Sep 17 03:23:58.001070 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:23:58.001038    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ae8e927_56ef_48da_951e_3f7e18968c68.slice/crio-3395c203940b115e3bbac8d8553e4bbe928bcdd47fc339700bcd880a5a56a429 WatchSource:0}: Error finding container 3395c203940b115e3bbac8d8553e4bbe928bcdd47fc339700bcd880a5a56a429: Status 404 returned error can't find the container with id 3395c203940b115e3bbac8d8553e4bbe928bcdd47fc339700bcd880a5a56a429
Sep 17 03:23:58.328303 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:58.328264    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:23:58.515123 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:23:58.515083    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d" event={"ID":"0ae8e927-56ef-48da-951e-3f7e18968c68","Type":"ContainerStarted","Data":"3395c203940b115e3bbac8d8553e4bbe928bcdd47fc339700bcd880a5a56a429"}
Sep 17 03:24:00.525307 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:00.525270    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d" event={"ID":"0ae8e927-56ef-48da-951e-3f7e18968c68","Type":"ContainerStarted","Data":"60b44c527a8abf422cc3c03461b55c3dc619cb767c3534ef76fe6c29d9a879ff"}
Sep 17 03:24:00.541150 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:00.541101    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6t6d" podStartSLOduration=1.7706578739999999 podStartE2EDuration="3.541084392s" totalImagesPullingTime="1.770426518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:23:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:24:00.539717241 +0000 UTC m=+268.884443320" watchObservedRunningTime="2026-09-17 03:24:00.541084392 +0000 UTC m=+268.885810471"
Sep 17 03:24:01.300272 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:01.300237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:01.403990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:01.403951    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:01.508582 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:01.508554    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-s22rg"
Sep 17 03:24:03.144407 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.144369    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"]
Sep 17 03:24:03.146577 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.146561    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"
Sep 17 03:24:03.148407 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.148375    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 03:24:03.148407 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.148395    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4svbt\""
Sep 17 03:24:03.148407 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.148376    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 03:24:03.159264 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.159236    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"]
Sep 17 03:24:03.228917 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.228876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f06ff9b-0c95-4e92-bc4f-343cc8671887-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc\" (UID: \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"
Sep 17 03:24:03.228917 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.228918    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8zdz\" (UniqueName: \"kubernetes.io/projected/6f06ff9b-0c95-4e92-bc4f-343cc8671887-kube-api-access-h8zdz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc\" (UID: \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"
Sep 17 03:24:03.229133 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.228952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f06ff9b-0c95-4e92-bc4f-343cc8671887-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc\" (UID: \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"
Sep 17 03:24:03.329332 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.329301    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f06ff9b-0c95-4e92-bc4f-343cc8671887-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc\" (UID: \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"
Sep 17 03:24:03.329536 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.329337    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8zdz\" (UniqueName: \"kubernetes.io/projected/6f06ff9b-0c95-4e92-bc4f-343cc8671887-kube-api-access-h8zdz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc\" (UID: \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"
Sep 17 03:24:03.329536 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.329372    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f06ff9b-0c95-4e92-bc4f-343cc8671887-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc\" (UID: \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"
Sep 17 03:24:03.329715 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.329697    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f06ff9b-0c95-4e92-bc4f-343cc8671887-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc\" (UID: \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"
Sep 17 03:24:03.329759 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.329721    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f06ff9b-0c95-4e92-bc4f-343cc8671887-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc\" (UID: \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"
Sep 17 03:24:03.338608 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.338585    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8zdz\" (UniqueName: \"kubernetes.io/projected/6f06ff9b-0c95-4e92-bc4f-343cc8671887-kube-api-access-h8zdz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc\" (UID: \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"
Sep 17 03:24:03.394708 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.394629    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:03.455943 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.455906    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"
Sep 17 03:24:03.721851 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:03.721827    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"]
Sep 17 03:24:03.723952 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:24:03.723922    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f06ff9b_0c95_4e92_bc4f_343cc8671887.slice/crio-bec158ab52c612d0bf1ab83cdab58e6922275deb73db25b4f235f165b5f05767 WatchSource:0}: Error finding container bec158ab52c612d0bf1ab83cdab58e6922275deb73db25b4f235f165b5f05767: Status 404 returned error can't find the container with id bec158ab52c612d0bf1ab83cdab58e6922275deb73db25b4f235f165b5f05767
Sep 17 03:24:04.542203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:04.542154    2576 generic.go:356] "Generic (PLEG): container finished" podID="6f06ff9b-0c95-4e92-bc4f-343cc8671887" containerID="780e5e89b1c8724e44ad905d95c3a5bd80c7e53d71217692e6ded614f03ac5de" exitCode=0
Sep 17 03:24:04.542614 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:04.542245    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc" event={"ID":"6f06ff9b-0c95-4e92-bc4f-343cc8671887","Type":"ContainerDied","Data":"780e5e89b1c8724e44ad905d95c3a5bd80c7e53d71217692e6ded614f03ac5de"}
Sep 17 03:24:04.542614 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:04.542300    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc" event={"ID":"6f06ff9b-0c95-4e92-bc4f-343cc8671887","Type":"ContainerStarted","Data":"bec158ab52c612d0bf1ab83cdab58e6922275deb73db25b4f235f165b5f05767"}
Sep 17 03:24:05.547930 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:05.547901    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc" event={"ID":"6f06ff9b-0c95-4e92-bc4f-343cc8671887","Type":"ContainerStarted","Data":"e05abc8e28e7276dbb9c87555ab4583c9dc1aa439482b5a19dbd6d41bb6b8291"}
Sep 17 03:24:06.553453 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:06.553398    2576 generic.go:356] "Generic (PLEG): container finished" podID="6f06ff9b-0c95-4e92-bc4f-343cc8671887" containerID="e05abc8e28e7276dbb9c87555ab4583c9dc1aa439482b5a19dbd6d41bb6b8291" exitCode=0
Sep 17 03:24:06.553831 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:06.553486    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc" event={"ID":"6f06ff9b-0c95-4e92-bc4f-343cc8671887","Type":"ContainerDied","Data":"e05abc8e28e7276dbb9c87555ab4583c9dc1aa439482b5a19dbd6d41bb6b8291"}
Sep 17 03:24:07.559394 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:07.559354    2576 generic.go:356] "Generic (PLEG): container finished" podID="6f06ff9b-0c95-4e92-bc4f-343cc8671887" containerID="16f5578018518f8f66cff10a9883022785687f7b075b49fe64e76b9413331a6a" exitCode=0
Sep 17 03:24:07.559811 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:07.559460    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc" event={"ID":"6f06ff9b-0c95-4e92-bc4f-343cc8671887","Type":"ContainerDied","Data":"16f5578018518f8f66cff10a9883022785687f7b075b49fe64e76b9413331a6a"}
Sep 17 03:24:08.683159 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:08.683136    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"
Sep 17 03:24:08.774962 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:08.774927    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8zdz\" (UniqueName: \"kubernetes.io/projected/6f06ff9b-0c95-4e92-bc4f-343cc8671887-kube-api-access-h8zdz\") pod \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\" (UID: \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\") "
Sep 17 03:24:08.775125 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:08.775005    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f06ff9b-0c95-4e92-bc4f-343cc8671887-util\") pod \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\" (UID: \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\") "
Sep 17 03:24:08.775186 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:08.775159    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f06ff9b-0c95-4e92-bc4f-343cc8671887-bundle\") pod \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\" (UID: \"6f06ff9b-0c95-4e92-bc4f-343cc8671887\") "
Sep 17 03:24:08.775847 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:08.775818    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f06ff9b-0c95-4e92-bc4f-343cc8671887-bundle" (OuterVolumeSpecName: "bundle") pod "6f06ff9b-0c95-4e92-bc4f-343cc8671887" (UID: "6f06ff9b-0c95-4e92-bc4f-343cc8671887"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:24:08.777093 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:08.777071    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f06ff9b-0c95-4e92-bc4f-343cc8671887-kube-api-access-h8zdz" (OuterVolumeSpecName: "kube-api-access-h8zdz") pod "6f06ff9b-0c95-4e92-bc4f-343cc8671887" (UID: "6f06ff9b-0c95-4e92-bc4f-343cc8671887"). InnerVolumeSpecName "kube-api-access-h8zdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:24:08.780392 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:08.780293    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f06ff9b-0c95-4e92-bc4f-343cc8671887-util" (OuterVolumeSpecName: "util") pod "6f06ff9b-0c95-4e92-bc4f-343cc8671887" (UID: "6f06ff9b-0c95-4e92-bc4f-343cc8671887"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:24:08.876153 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:08.876046    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f06ff9b-0c95-4e92-bc4f-343cc8671887-util\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:24:08.876153 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:08.876091    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f06ff9b-0c95-4e92-bc4f-343cc8671887-bundle\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:24:08.876153 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:08.876104    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8zdz\" (UniqueName: \"kubernetes.io/projected/6f06ff9b-0c95-4e92-bc4f-343cc8671887-kube-api-access-h8zdz\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:24:09.316680 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:09.316640    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:09.362212 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:09.362178    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:09.569285 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:09.569196    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc" event={"ID":"6f06ff9b-0c95-4e92-bc4f-343cc8671887","Type":"ContainerDied","Data":"bec158ab52c612d0bf1ab83cdab58e6922275deb73db25b4f235f165b5f05767"}
Sep 17 03:24:09.569285 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:09.569239    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5lwbc"
Sep 17 03:24:09.569285 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:09.569249    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bec158ab52c612d0bf1ab83cdab58e6922275deb73db25b4f235f165b5f05767"
Sep 17 03:24:18.731632 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:18.731589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:18.829845 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:18.829800    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:19.353700 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.353666    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:19.546211 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.546175    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"]
Sep 17 03:24:19.546551 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.546534    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f06ff9b-0c95-4e92-bc4f-343cc8671887" containerName="util"
Sep 17 03:24:19.546605 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.546555    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f06ff9b-0c95-4e92-bc4f-343cc8671887" containerName="util"
Sep 17 03:24:19.546605 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.546565    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f06ff9b-0c95-4e92-bc4f-343cc8671887" containerName="extract"
Sep 17 03:24:19.546605 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.546572    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f06ff9b-0c95-4e92-bc4f-343cc8671887" containerName="extract"
Sep 17 03:24:19.546605 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.546596    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f06ff9b-0c95-4e92-bc4f-343cc8671887" containerName="pull"
Sep 17 03:24:19.546605 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.546604    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f06ff9b-0c95-4e92-bc4f-343cc8671887" containerName="pull"
Sep 17 03:24:19.546751 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.546670    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f06ff9b-0c95-4e92-bc4f-343cc8671887" containerName="extract"
Sep 17 03:24:19.549607 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.549574    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"
Sep 17 03:24:19.551766 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.551742    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 03:24:19.552217 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.552198    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 03:24:19.552284 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.552197    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4svbt\""
Sep 17 03:24:19.556918 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.556859    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"]
Sep 17 03:24:19.593848 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.593816    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:19.659940 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.659857    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:19.663301 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.663278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8\" (UID: \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"
Sep 17 03:24:19.663386 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.663332    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d86n\" (UniqueName: \"kubernetes.io/projected/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-kube-api-access-8d86n\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8\" (UID: \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"
Sep 17 03:24:19.663420 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.663394    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8\" (UID: \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"
Sep 17 03:24:19.764658 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.764609    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d86n\" (UniqueName: \"kubernetes.io/projected/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-kube-api-access-8d86n\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8\" (UID: \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"
Sep 17 03:24:19.765321 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.764686    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8\" (UID: \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"
Sep 17 03:24:19.765321 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.764714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8\" (UID: \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"
Sep 17 03:24:19.765321 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.765153    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8\" (UID: \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"
Sep 17 03:24:19.765321 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.765223    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8\" (UID: \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"
Sep 17 03:24:19.767132 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.767107    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"]
Sep 17 03:24:19.769380 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.769360    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.771538 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.771517    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 17 03:24:19.771659 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.771642    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 17 03:24:19.771807 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.771769    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8vv95\""
Sep 17 03:24:19.771910 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.771811    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 17 03:24:19.771910 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.771870    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 17 03:24:19.779955 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.779930    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d86n\" (UniqueName: \"kubernetes.io/projected/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-kube-api-access-8d86n\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8\" (UID: \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"
Sep 17 03:24:19.791100 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.791074    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"]
Sep 17 03:24:19.861321 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.861286    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"
Sep 17 03:24:19.865335 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.865318    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fb7347b-fab7-4d3f-a416-2943865f87b3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.865414 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.865345    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fb7347b-fab7-4d3f-a416-2943865f87b3-tmp\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.865414 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.865366    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9fb7347b-fab7-4d3f-a416-2943865f87b3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.865414 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.865403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r686w\" (UniqueName: \"kubernetes.io/projected/9fb7347b-fab7-4d3f-a416-2943865f87b3-kube-api-access-r686w\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.865575 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.865508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9fb7347b-fab7-4d3f-a416-2943865f87b3-manifests\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.879523 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.879487    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:19.966792 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.966753    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fb7347b-fab7-4d3f-a416-2943865f87b3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.966792 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.966794    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fb7347b-fab7-4d3f-a416-2943865f87b3-tmp\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.967047 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.966814    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9fb7347b-fab7-4d3f-a416-2943865f87b3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.967047 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.966841    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r686w\" (UniqueName: \"kubernetes.io/projected/9fb7347b-fab7-4d3f-a416-2943865f87b3-kube-api-access-r686w\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.967047 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.966860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9fb7347b-fab7-4d3f-a416-2943865f87b3-manifests\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.967302 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.967275    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9fb7347b-fab7-4d3f-a416-2943865f87b3-manifests\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.969300 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.969273    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fb7347b-fab7-4d3f-a416-2943865f87b3-tmp\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.969446 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.969418    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fb7347b-fab7-4d3f-a416-2943865f87b3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.969542 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.969521    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9fb7347b-fab7-4d3f-a416-2943865f87b3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.974597 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.974573    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r686w\" (UniqueName: \"kubernetes.io/projected/9fb7347b-fab7-4d3f-a416-2943865f87b3-kube-api-access-r686w\") pod \"opendatahub-operator-controller-manager-6977f7b7bc-hs6g2\" (UID: \"9fb7347b-fab7-4d3f-a416-2943865f87b3\") " pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:19.979313 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:19.979286    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:20.079607 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:20.079566    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:20.081774 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:20.081750    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:20.132078 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:20.132002    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"]
Sep 17 03:24:20.137049 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:24:20.137012    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb36c7a_cc2f_4e0d_a51e_5e24b4932021.slice/crio-fdef7e7241a5d29e77d7e5c1b6a5dbd792888d2447a036c104a3113326580f9c WatchSource:0}: Error finding container fdef7e7241a5d29e77d7e5c1b6a5dbd792888d2447a036c104a3113326580f9c: Status 404 returned error can't find the container with id fdef7e7241a5d29e77d7e5c1b6a5dbd792888d2447a036c104a3113326580f9c
Sep 17 03:24:20.187702 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:20.187672    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:20.295894 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:20.295863    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:20.354232 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:20.354204    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"]
Sep 17 03:24:20.356553 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:24:20.356525    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb7347b_fab7_4d3f_a416_2943865f87b3.slice/crio-83c4091eca916165e0e36f6bc016ab7f2446c340934201f2358e29ab5c3f004b WatchSource:0}: Error finding container 83c4091eca916165e0e36f6bc016ab7f2446c340934201f2358e29ab5c3f004b: Status 404 returned error can't find the container with id 83c4091eca916165e0e36f6bc016ab7f2446c340934201f2358e29ab5c3f004b
Sep 17 03:24:20.374309 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:20.374267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:20.614127 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:20.614090    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2" event={"ID":"9fb7347b-fab7-4d3f-a416-2943865f87b3","Type":"ContainerStarted","Data":"83c4091eca916165e0e36f6bc016ab7f2446c340934201f2358e29ab5c3f004b"}
Sep 17 03:24:20.615575 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:20.615547    2576 generic.go:356] "Generic (PLEG): container finished" podID="2cb36c7a-cc2f-4e0d-a51e-5e24b4932021" containerID="913db6c0e0b998b79ea5bdc1da94021f28728a14fbe30031d2d62b8f7db99d79" exitCode=0
Sep 17 03:24:20.615709 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:20.615632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8" event={"ID":"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021","Type":"ContainerDied","Data":"913db6c0e0b998b79ea5bdc1da94021f28728a14fbe30031d2d62b8f7db99d79"}
Sep 17 03:24:20.615709 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:20.615686    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8" event={"ID":"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021","Type":"ContainerStarted","Data":"fdef7e7241a5d29e77d7e5c1b6a5dbd792888d2447a036c104a3113326580f9c"}
Sep 17 03:24:21.623444 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:21.623391    2576 generic.go:356] "Generic (PLEG): container finished" podID="2cb36c7a-cc2f-4e0d-a51e-5e24b4932021" containerID="5b95c733772fa585a0138076d03ace82897902bae7b35ce00ea84b46b5ad97a4" exitCode=0
Sep 17 03:24:21.624021 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:21.623471    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8" event={"ID":"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021","Type":"ContainerDied","Data":"5b95c733772fa585a0138076d03ace82897902bae7b35ce00ea84b46b5ad97a4"}
Sep 17 03:24:23.634361 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:23.634324    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2" event={"ID":"9fb7347b-fab7-4d3f-a416-2943865f87b3","Type":"ContainerStarted","Data":"31b6d8184aea4bc849472475a0467a785e12bfe34adf8608c11913072e439944"}
Sep 17 03:24:23.636670 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:23.636643    2576 generic.go:356] "Generic (PLEG): container finished" podID="2cb36c7a-cc2f-4e0d-a51e-5e24b4932021" containerID="6c3251f835c35fee5da3ee32eb9a0fd67cf9734db1e96a943ccbed028fdc5e0a" exitCode=0
Sep 17 03:24:23.636799 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:23.636720    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8" event={"ID":"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021","Type":"ContainerDied","Data":"6c3251f835c35fee5da3ee32eb9a0fd67cf9734db1e96a943ccbed028fdc5e0a"}
Sep 17 03:24:24.642538 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:24.642497    2576 generic.go:356] "Generic (PLEG): container finished" podID="9fb7347b-fab7-4d3f-a416-2943865f87b3" containerID="31b6d8184aea4bc849472475a0467a785e12bfe34adf8608c11913072e439944" exitCode=0
Sep 17 03:24:24.642971 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:24.642587    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2" event={"ID":"9fb7347b-fab7-4d3f-a416-2943865f87b3","Type":"ContainerDied","Data":"31b6d8184aea4bc849472475a0467a785e12bfe34adf8608c11913072e439944"}
Sep 17 03:24:24.766451 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:24.766415    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"
Sep 17 03:24:24.910399 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:24.910323    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-bundle\") pod \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\" (UID: \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\") "
Sep 17 03:24:24.910399 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:24.910354    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8d86n\" (UniqueName: \"kubernetes.io/projected/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-kube-api-access-8d86n\") pod \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\" (UID: \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\") "
Sep 17 03:24:24.910399 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:24.910392    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-util\") pod \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\" (UID: \"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021\") "
Sep 17 03:24:24.911070 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:24.911043    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-bundle" (OuterVolumeSpecName: "bundle") pod "2cb36c7a-cc2f-4e0d-a51e-5e24b4932021" (UID: "2cb36c7a-cc2f-4e0d-a51e-5e24b4932021"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:24:24.912476 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:24.912454    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-kube-api-access-8d86n" (OuterVolumeSpecName: "kube-api-access-8d86n") pod "2cb36c7a-cc2f-4e0d-a51e-5e24b4932021" (UID: "2cb36c7a-cc2f-4e0d-a51e-5e24b4932021"). InnerVolumeSpecName "kube-api-access-8d86n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:24:24.915802 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:24.915680    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-util" (OuterVolumeSpecName: "util") pod "2cb36c7a-cc2f-4e0d-a51e-5e24b4932021" (UID: "2cb36c7a-cc2f-4e0d-a51e-5e24b4932021"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:24:25.011592 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:25.011559    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-util\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:24:25.011592 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:25.011592    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-bundle\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:24:25.011746 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:25.011607    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d86n\" (UniqueName: \"kubernetes.io/projected/2cb36c7a-cc2f-4e0d-a51e-5e24b4932021-kube-api-access-8d86n\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:24:25.648665 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:25.648579    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2" event={"ID":"9fb7347b-fab7-4d3f-a416-2943865f87b3","Type":"ContainerStarted","Data":"2283593346e8565835f4c45942f78116a36342eccde6a1212f080c373ec05044"}
Sep 17 03:24:25.648665 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:25.648658    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:25.650747 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:25.650723    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8"
Sep 17 03:24:25.650747 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:25.650739    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0s9hm8" event={"ID":"2cb36c7a-cc2f-4e0d-a51e-5e24b4932021","Type":"ContainerDied","Data":"fdef7e7241a5d29e77d7e5c1b6a5dbd792888d2447a036c104a3113326580f9c"}
Sep 17 03:24:25.650896 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:25.650781    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fdef7e7241a5d29e77d7e5c1b6a5dbd792888d2447a036c104a3113326580f9c"
Sep 17 03:24:25.669361 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:25.669315    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2" podStartSLOduration=2.535909098 podStartE2EDuration="6.669299152s" totalImagesPullingTime="3.084091602s" totalInitContainerRuntime="1.049298452s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:24:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:24:25.668142721 +0000 UTC m=+294.012868798" watchObservedRunningTime="2026-09-17 03:24:25.669299152 +0000 UTC m=+294.014025231"
Sep 17 03:24:29.122604 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:29.122562    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:30.577038 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:30.577002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:32.147522 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:32.147493    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:24:32.150983 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:32.150948    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:24:32.154240 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:32.154215    2576 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 03:24:36.658877 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:36.658845    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6977f7b7bc-hs6g2"
Sep 17 03:24:36.833563 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:36.833528    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:36.888505 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:36.888476    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:38.782655 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:38.782618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:38.906913 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:38.906875    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:39.048166 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.048079    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:39.110030 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.109987    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"]
Sep 17 03:24:39.110297 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.110283    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb36c7a-cc2f-4e0d-a51e-5e24b4932021" containerName="util"
Sep 17 03:24:39.110297 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.110298    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb36c7a-cc2f-4e0d-a51e-5e24b4932021" containerName="util"
Sep 17 03:24:39.110416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.110313    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb36c7a-cc2f-4e0d-a51e-5e24b4932021" containerName="pull"
Sep 17 03:24:39.110416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.110319    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb36c7a-cc2f-4e0d-a51e-5e24b4932021" containerName="pull"
Sep 17 03:24:39.110416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.110326    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb36c7a-cc2f-4e0d-a51e-5e24b4932021" containerName="extract"
Sep 17 03:24:39.110416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.110331    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb36c7a-cc2f-4e0d-a51e-5e24b4932021" containerName="extract"
Sep 17 03:24:39.110416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.110374    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cb36c7a-cc2f-4e0d-a51e-5e24b4932021" containerName="extract"
Sep 17 03:24:39.112281 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.112262    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"
Sep 17 03:24:39.114414 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.114388    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 17 03:24:39.114566 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.114453    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 17 03:24:39.114566 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.114489    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 17 03:24:39.114566 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.114540    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 17 03:24:39.114966 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.114952    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fwmr2\""
Sep 17 03:24:39.123060 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.123041    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"]
Sep 17 03:24:39.213357 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.213323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/457fad5e-53c6-439b-9000-859da48c58db-tmp\") pod \"kube-auth-proxy-5bc6dc7888-57x9k\" (UID: \"457fad5e-53c6-439b-9000-859da48c58db\") " pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"
Sep 17 03:24:39.213357 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.213360    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccb2f\" (UniqueName: \"kubernetes.io/projected/457fad5e-53c6-439b-9000-859da48c58db-kube-api-access-ccb2f\") pod \"kube-auth-proxy-5bc6dc7888-57x9k\" (UID: \"457fad5e-53c6-439b-9000-859da48c58db\") " pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"
Sep 17 03:24:39.213584 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.213410    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/457fad5e-53c6-439b-9000-859da48c58db-tls-certs\") pod \"kube-auth-proxy-5bc6dc7888-57x9k\" (UID: \"457fad5e-53c6-439b-9000-859da48c58db\") " pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"
Sep 17 03:24:39.314488 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.314451    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/457fad5e-53c6-439b-9000-859da48c58db-tmp\") pod \"kube-auth-proxy-5bc6dc7888-57x9k\" (UID: \"457fad5e-53c6-439b-9000-859da48c58db\") " pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"
Sep 17 03:24:39.314640 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.314494    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccb2f\" (UniqueName: \"kubernetes.io/projected/457fad5e-53c6-439b-9000-859da48c58db-kube-api-access-ccb2f\") pod \"kube-auth-proxy-5bc6dc7888-57x9k\" (UID: \"457fad5e-53c6-439b-9000-859da48c58db\") " pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"
Sep 17 03:24:39.314640 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.314627    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/457fad5e-53c6-439b-9000-859da48c58db-tls-certs\") pod \"kube-auth-proxy-5bc6dc7888-57x9k\" (UID: \"457fad5e-53c6-439b-9000-859da48c58db\") " pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"
Sep 17 03:24:39.314801 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:24:39.314778    2576 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 17 03:24:39.314887 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:24:39.314874    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/457fad5e-53c6-439b-9000-859da48c58db-tls-certs podName:457fad5e-53c6-439b-9000-859da48c58db nodeName:}" failed. No retries permitted until 2026-09-17 03:24:39.814851582 +0000 UTC m=+308.159577653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/457fad5e-53c6-439b-9000-859da48c58db-tls-certs") pod "kube-auth-proxy-5bc6dc7888-57x9k" (UID: "457fad5e-53c6-439b-9000-859da48c58db") : secret "kube-auth-proxy-tls" not found
Sep 17 03:24:39.316764 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.316745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/457fad5e-53c6-439b-9000-859da48c58db-tmp\") pod \"kube-auth-proxy-5bc6dc7888-57x9k\" (UID: \"457fad5e-53c6-439b-9000-859da48c58db\") " pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"
Sep 17 03:24:39.324898 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.324871    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccb2f\" (UniqueName: \"kubernetes.io/projected/457fad5e-53c6-439b-9000-859da48c58db-kube-api-access-ccb2f\") pod \"kube-auth-proxy-5bc6dc7888-57x9k\" (UID: \"457fad5e-53c6-439b-9000-859da48c58db\") " pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"
Sep 17 03:24:39.491944 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.491911    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:39.819371 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.819327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/457fad5e-53c6-439b-9000-859da48c58db-tls-certs\") pod \"kube-auth-proxy-5bc6dc7888-57x9k\" (UID: \"457fad5e-53c6-439b-9000-859da48c58db\") " pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"
Sep 17 03:24:39.821745 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:39.821720    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/457fad5e-53c6-439b-9000-859da48c58db-tls-certs\") pod \"kube-auth-proxy-5bc6dc7888-57x9k\" (UID: \"457fad5e-53c6-439b-9000-859da48c58db\") " pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"
Sep 17 03:24:40.021829 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:40.021794    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"
Sep 17 03:24:40.294183 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:40.294159    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k"]
Sep 17 03:24:40.296147 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:24:40.296111    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod457fad5e_53c6_439b_9000_859da48c58db.slice/crio-59947ddada9b8b8d5dfa1f85d5f0a46639dc6d63aa5fa14319f801dc0ad08652 WatchSource:0}: Error finding container 59947ddada9b8b8d5dfa1f85d5f0a46639dc6d63aa5fa14319f801dc0ad08652: Status 404 returned error can't find the container with id 59947ddada9b8b8d5dfa1f85d5f0a46639dc6d63aa5fa14319f801dc0ad08652
Sep 17 03:24:40.297952 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:40.297933    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 03:24:40.317698 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:40.317666    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:40.384081 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:40.384001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:40.716655 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:40.716579    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k" event={"ID":"457fad5e-53c6-439b-9000-859da48c58db","Type":"ContainerStarted","Data":"59947ddada9b8b8d5dfa1f85d5f0a46639dc6d63aa5fa14319f801dc0ad08652"}
Sep 17 03:24:41.312548 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:41.312511    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:43.732792 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:43.732751    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k" event={"ID":"457fad5e-53c6-439b-9000-859da48c58db","Type":"ContainerStarted","Data":"7cf99c8a04eda660d6fcc7b24010cbe8e3859a9464e3b5434672af858efd6e52"}
Sep 17 03:24:43.746924 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:43.746866    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5bc6dc7888-57x9k" podStartSLOduration=2.207895215 podStartE2EDuration="4.746845251s" totalImagesPullingTime="2.538950036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:24:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:24:43.746409392 +0000 UTC m=+312.091135462" watchObservedRunningTime="2026-09-17 03:24:43.746845251 +0000 UTC m=+312.091571331"
Sep 17 03:24:45.023582 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:45.023547    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:45.077804 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:45.077767    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:45.303612 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:45.303584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:45.487396 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:45.487359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:45.587476 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:45.587349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:49.890279 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:49.890249    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-h64t5"]
Sep 17 03:24:49.892361 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:49.892346    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-h64t5"
Sep 17 03:24:49.894343 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:49.894313    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 17 03:24:49.894466 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:49.894366    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-hr5np\""
Sep 17 03:24:49.903067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:49.903043    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-h64t5"]
Sep 17 03:24:49.935900 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:49.935869    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:49.992227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:49.992193    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:49.998722 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:49.998692    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfpxt\" (UniqueName: \"kubernetes.io/projected/d784a82c-aa35-4321-89ef-de3cee39c699-kube-api-access-gfpxt\") pod \"odh-model-controller-7998c67dc4-h64t5\" (UID: \"d784a82c-aa35-4321-89ef-de3cee39c699\") " pod="opendatahub/odh-model-controller-7998c67dc4-h64t5"
Sep 17 03:24:49.998877 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:49.998751    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d784a82c-aa35-4321-89ef-de3cee39c699-cert\") pod \"odh-model-controller-7998c67dc4-h64t5\" (UID: \"d784a82c-aa35-4321-89ef-de3cee39c699\") " pod="opendatahub/odh-model-controller-7998c67dc4-h64t5"
Sep 17 03:24:50.093217 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.093187    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:50.103690 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.099745    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d784a82c-aa35-4321-89ef-de3cee39c699-cert\") pod \"odh-model-controller-7998c67dc4-h64t5\" (UID: \"d784a82c-aa35-4321-89ef-de3cee39c699\") " pod="opendatahub/odh-model-controller-7998c67dc4-h64t5"
Sep 17 03:24:50.103690 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.099867    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfpxt\" (UniqueName: \"kubernetes.io/projected/d784a82c-aa35-4321-89ef-de3cee39c699-kube-api-access-gfpxt\") pod \"odh-model-controller-7998c67dc4-h64t5\" (UID: \"d784a82c-aa35-4321-89ef-de3cee39c699\") " pod="opendatahub/odh-model-controller-7998c67dc4-h64t5"
Sep 17 03:24:50.103690 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:24:50.100368    2576 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 17 03:24:50.103958 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:24:50.103819    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d784a82c-aa35-4321-89ef-de3cee39c699-cert podName:d784a82c-aa35-4321-89ef-de3cee39c699 nodeName:}" failed. No retries permitted until 2026-09-17 03:24:50.603761518 +0000 UTC m=+318.948487640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d784a82c-aa35-4321-89ef-de3cee39c699-cert") pod "odh-model-controller-7998c67dc4-h64t5" (UID: "d784a82c-aa35-4321-89ef-de3cee39c699") : secret "odh-model-controller-webhook-cert" not found
Sep 17 03:24:50.109552 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.109522    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfpxt\" (UniqueName: \"kubernetes.io/projected/d784a82c-aa35-4321-89ef-de3cee39c699-kube-api-access-gfpxt\") pod \"odh-model-controller-7998c67dc4-h64t5\" (UID: \"d784a82c-aa35-4321-89ef-de3cee39c699\") " pod="opendatahub/odh-model-controller-7998c67dc4-h64t5"
Sep 17 03:24:50.193707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.193621    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:50.444012 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.443932    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:50.460699 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.460661    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fqsqm"]
Sep 17 03:24:50.463005 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.462986    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm"
Sep 17 03:24:50.464947 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.464926    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 17 03:24:50.475125 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.475096    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fqsqm"]
Sep 17 03:24:50.541785 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.541750    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:50.603567 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.603528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpbhc\" (UniqueName: \"kubernetes.io/projected/071b4aba-067f-47ab-b292-3234865c0eb3-kube-api-access-lpbhc\") pod \"model-serving-api-54d89bd5fc-fqsqm\" (UID: \"071b4aba-067f-47ab-b292-3234865c0eb3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm"
Sep 17 03:24:50.603746 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.603583    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/071b4aba-067f-47ab-b292-3234865c0eb3-tls-certs\") pod \"model-serving-api-54d89bd5fc-fqsqm\" (UID: \"071b4aba-067f-47ab-b292-3234865c0eb3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm"
Sep 17 03:24:50.705067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.704970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/071b4aba-067f-47ab-b292-3234865c0eb3-tls-certs\") pod \"model-serving-api-54d89bd5fc-fqsqm\" (UID: \"071b4aba-067f-47ab-b292-3234865c0eb3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm"
Sep 17 03:24:50.705219 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.705068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d784a82c-aa35-4321-89ef-de3cee39c699-cert\") pod \"odh-model-controller-7998c67dc4-h64t5\" (UID: \"d784a82c-aa35-4321-89ef-de3cee39c699\") " pod="opendatahub/odh-model-controller-7998c67dc4-h64t5"
Sep 17 03:24:50.705219 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.705093    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbhc\" (UniqueName: \"kubernetes.io/projected/071b4aba-067f-47ab-b292-3234865c0eb3-kube-api-access-lpbhc\") pod \"model-serving-api-54d89bd5fc-fqsqm\" (UID: \"071b4aba-067f-47ab-b292-3234865c0eb3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm"
Sep 17 03:24:50.705219 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:24:50.705170    2576 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 17 03:24:50.705319 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:24:50.705268    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/071b4aba-067f-47ab-b292-3234865c0eb3-tls-certs podName:071b4aba-067f-47ab-b292-3234865c0eb3 nodeName:}" failed. No retries permitted until 2026-09-17 03:24:51.205247965 +0000 UTC m=+319.549974042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/071b4aba-067f-47ab-b292-3234865c0eb3-tls-certs") pod "model-serving-api-54d89bd5fc-fqsqm" (UID: "071b4aba-067f-47ab-b292-3234865c0eb3") : secret "model-serving-api-tls" not found
Sep 17 03:24:50.707554 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.707533    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d784a82c-aa35-4321-89ef-de3cee39c699-cert\") pod \"odh-model-controller-7998c67dc4-h64t5\" (UID: \"d784a82c-aa35-4321-89ef-de3cee39c699\") " pod="opendatahub/odh-model-controller-7998c67dc4-h64t5"
Sep 17 03:24:50.715156 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.715131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpbhc\" (UniqueName: \"kubernetes.io/projected/071b4aba-067f-47ab-b292-3234865c0eb3-kube-api-access-lpbhc\") pod \"model-serving-api-54d89bd5fc-fqsqm\" (UID: \"071b4aba-067f-47ab-b292-3234865c0eb3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm"
Sep 17 03:24:50.802680 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.802639    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-h64t5"
Sep 17 03:24:50.829664 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:50.829630    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:51.088264 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:51.088235    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-h64t5"]
Sep 17 03:24:51.092660 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:24:51.092627    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd784a82c_aa35_4321_89ef_de3cee39c699.slice/crio-5fc524251ec3170d440e186f173212c5130a1d9b9990018a8c9f75a14c6bc8dd WatchSource:0}: Error finding container 5fc524251ec3170d440e186f173212c5130a1d9b9990018a8c9f75a14c6bc8dd: Status 404 returned error can't find the container with id 5fc524251ec3170d440e186f173212c5130a1d9b9990018a8c9f75a14c6bc8dd
Sep 17 03:24:51.208906 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:51.208864    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/071b4aba-067f-47ab-b292-3234865c0eb3-tls-certs\") pod \"model-serving-api-54d89bd5fc-fqsqm\" (UID: \"071b4aba-067f-47ab-b292-3234865c0eb3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm"
Sep 17 03:24:51.211549 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:51.211511    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/071b4aba-067f-47ab-b292-3234865c0eb3-tls-certs\") pod \"model-serving-api-54d89bd5fc-fqsqm\" (UID: \"071b4aba-067f-47ab-b292-3234865c0eb3\") " pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm"
Sep 17 03:24:51.372727 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:51.372630    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm"
Sep 17 03:24:51.645179 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:51.645153    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fqsqm"]
Sep 17 03:24:51.647507 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:24:51.647468    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod071b4aba_067f_47ab_b292_3234865c0eb3.slice/crio-91a23f4e1defa31821c1a7d5ab604b06e7137a6b33b61ce8a1a33f8b22eca42c WatchSource:0}: Error finding container 91a23f4e1defa31821c1a7d5ab604b06e7137a6b33b61ce8a1a33f8b22eca42c: Status 404 returned error can't find the container with id 91a23f4e1defa31821c1a7d5ab604b06e7137a6b33b61ce8a1a33f8b22eca42c
Sep 17 03:24:51.769053 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:51.769009    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm" event={"ID":"071b4aba-067f-47ab-b292-3234865c0eb3","Type":"ContainerStarted","Data":"91a23f4e1defa31821c1a7d5ab604b06e7137a6b33b61ce8a1a33f8b22eca42c"}
Sep 17 03:24:51.770657 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:51.770612    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-h64t5" event={"ID":"d784a82c-aa35-4321-89ef-de3cee39c699","Type":"ContainerStarted","Data":"5fc524251ec3170d440e186f173212c5130a1d9b9990018a8c9f75a14c6bc8dd"}
Sep 17 03:24:54.787003 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:54.786967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm" event={"ID":"071b4aba-067f-47ab-b292-3234865c0eb3","Type":"ContainerStarted","Data":"c689b550ba51965ca5ccdd55ef9db3e6ad76834baf9ad5568dc58a8110b0eec5"}
Sep 17 03:24:54.787507 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:54.787076    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm"
Sep 17 03:24:54.788598 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:54.788575    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-h64t5" event={"ID":"d784a82c-aa35-4321-89ef-de3cee39c699","Type":"ContainerStarted","Data":"d13672775ce32598c22ce61e62e6d0811b6cda9eef386ff7ce6a90c5a29dc3f8"}
Sep 17 03:24:54.788703 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:54.788688    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-h64t5"
Sep 17 03:24:54.801080 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:54.801029    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm" podStartSLOduration=2.449010046 podStartE2EDuration="4.801014601s" totalImagesPullingTime="2.352004555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:24:54.800238818 +0000 UTC m=+323.144964897" watchObservedRunningTime="2026-09-17 03:24:54.801014601 +0000 UTC m=+323.145740682"
Sep 17 03:24:54.814345 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:54.814302    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-h64t5" podStartSLOduration=2.902006346 podStartE2EDuration="5.814289506s" totalImagesPullingTime="2.91228316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:24:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:24:54.813703969 +0000 UTC m=+323.158430049" watchObservedRunningTime="2026-09-17 03:24:54.814289506 +0000 UTC m=+323.159015584"
Sep 17 03:24:56.240221 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.240192    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"]
Sep 17 03:24:56.310512 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.310475    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"]
Sep 17 03:24:56.310694 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.310622    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.314624 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.314603    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 17 03:24:56.315325 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.315307    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 17 03:24:56.315454 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.315405    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 17 03:24:56.315536 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.315521    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4czlc\""
Sep 17 03:24:56.315626 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.315592    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 17 03:24:56.351425 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.351399    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70731ed8-7be6-45c7-b277-24e63a8ab980-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.351593 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.351500    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sbqb\" (UniqueName: \"kubernetes.io/projected/70731ed8-7be6-45c7-b277-24e63a8ab980-kube-api-access-8sbqb\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.351593 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.351540    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/70731ed8-7be6-45c7-b277-24e63a8ab980-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.351593 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.351559    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/70731ed8-7be6-45c7-b277-24e63a8ab980-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.351810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.351607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70731ed8-7be6-45c7-b277-24e63a8ab980-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.351810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.351633    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/70731ed8-7be6-45c7-b277-24e63a8ab980-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.351810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.351665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/70731ed8-7be6-45c7-b277-24e63a8ab980-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.452421 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.452381    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/70731ed8-7be6-45c7-b277-24e63a8ab980-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.452621 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.452456    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70731ed8-7be6-45c7-b277-24e63a8ab980-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.452621 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.452486    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/70731ed8-7be6-45c7-b277-24e63a8ab980-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.452621 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.452511    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/70731ed8-7be6-45c7-b277-24e63a8ab980-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.452621 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.452542    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70731ed8-7be6-45c7-b277-24e63a8ab980-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.452621 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.452566    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sbqb\" (UniqueName: \"kubernetes.io/projected/70731ed8-7be6-45c7-b277-24e63a8ab980-kube-api-access-8sbqb\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.452621 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.452612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/70731ed8-7be6-45c7-b277-24e63a8ab980-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.453465 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.453417    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/70731ed8-7be6-45c7-b277-24e63a8ab980-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.454913 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.454883    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/70731ed8-7be6-45c7-b277-24e63a8ab980-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.455008 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.454885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/70731ed8-7be6-45c7-b277-24e63a8ab980-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.455049 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.454998    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/70731ed8-7be6-45c7-b277-24e63a8ab980-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.455084 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.455047    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/70731ed8-7be6-45c7-b277-24e63a8ab980-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.460850 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.460828    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sbqb\" (UniqueName: \"kubernetes.io/projected/70731ed8-7be6-45c7-b277-24e63a8ab980-kube-api-access-8sbqb\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.461220 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.461204    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70731ed8-7be6-45c7-b277-24e63a8ab980-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-qwlft\" (UID: \"70731ed8-7be6-45c7-b277-24e63a8ab980\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.620461 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.620413    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:24:56.913555 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:56.913513    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"]
Sep 17 03:24:56.913911 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:24:56.913885    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70731ed8_7be6_45c7_b277_24e63a8ab980.slice/crio-913c9da48cde92babda49970f20928ffbd3df68d3f9be9ca6aed243fd85efe71 WatchSource:0}: Error finding container 913c9da48cde92babda49970f20928ffbd3df68d3f9be9ca6aed243fd85efe71: Status 404 returned error can't find the container with id 913c9da48cde92babda49970f20928ffbd3df68d3f9be9ca6aed243fd85efe71
Sep 17 03:24:57.803327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:57.803284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft" event={"ID":"70731ed8-7be6-45c7-b277-24e63a8ab980","Type":"ContainerStarted","Data":"913c9da48cde92babda49970f20928ffbd3df68d3f9be9ca6aed243fd85efe71"}
Sep 17 03:24:58.750808 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:58.750706    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:58.859909 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:58.859869    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:59.052143 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:59.052109    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 03:24:59.052209 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:59.052178    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 03:24:59.243212 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:59.243178    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:59.450741 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:59.450704    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:24:59.814627 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:59.814587    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft" event={"ID":"70731ed8-7be6-45c7-b277-24e63a8ab980","Type":"ContainerStarted","Data":"68bfa1d98a4a12c1c219db9278694ff448bd2b949443fcc93a7722ff4eb6429b"}
Sep 17 03:24:59.814795 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:24:59.814649    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:25:00.116810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:00.116717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:25:00.230017 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:00.229980    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:25:00.352509 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:00.352478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:25:00.821512 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:00.821483    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft"
Sep 17 03:25:00.828005 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:00.827970    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:25:00.848307 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:00.848245    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qwlft" podStartSLOduration=2.712626168 podStartE2EDuration="4.848229462s" totalImagesPullingTime="2.135603294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:24:59.852117807 +0000 UTC m=+328.196843887" watchObservedRunningTime="2026-09-17 03:25:00.848229462 +0000 UTC m=+329.192955542"
Sep 17 03:25:00.923101 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:00.923067    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:25:02.455448 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:02.455394    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:25:02.557083 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:02.557019    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:25:05.794877 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:05.794844    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-h64t5"
Sep 17 03:25:05.796641 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:05.796623    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-fqsqm"
Sep 17 03:25:45.054030 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:45.053991    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:25:45.110632 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:45.110591    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:25:47.002264 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:47.002227    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:25:47.102830 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:47.102799    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:25:50.882978 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:50.882940    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:25:50.981492 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:50.981455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:25:55.564621 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.564585    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"]
Sep 17 03:25:55.567122 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.567107    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"
Sep 17 03:25:55.569727 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.569702    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 03:25:55.570167 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.570146    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4svbt\""
Sep 17 03:25:55.570291 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.570190    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 03:25:55.577743 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.577717    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"]
Sep 17 03:25:55.665885 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.665843    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"]
Sep 17 03:25:55.668112 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.668096    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"
Sep 17 03:25:55.678528 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.678502    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"]
Sep 17 03:25:55.713002 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.712961    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmtcl\" (UniqueName: \"kubernetes.io/projected/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-kube-api-access-gmtcl\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg\" (UID: \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"
Sep 17 03:25:55.713002 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.713005    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg\" (UID: \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"
Sep 17 03:25:55.713240 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.713062    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg\" (UID: \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"
Sep 17 03:25:55.765966 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.765928    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"]
Sep 17 03:25:55.768519 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.768490    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"
Sep 17 03:25:55.776372 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.776340    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"]
Sep 17 03:25:55.814295 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.814256    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/857bc527-8cca-423d-b226-621d7fc41e37-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr\" (UID: \"857bc527-8cca-423d-b226-621d7fc41e37\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"
Sep 17 03:25:55.814295 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.814296    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg\" (UID: \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"
Sep 17 03:25:55.814572 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.814318    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/857bc527-8cca-423d-b226-621d7fc41e37-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr\" (UID: \"857bc527-8cca-423d-b226-621d7fc41e37\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"
Sep 17 03:25:55.814572 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.814488    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5b9k\" (UniqueName: \"kubernetes.io/projected/857bc527-8cca-423d-b226-621d7fc41e37-kube-api-access-d5b9k\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr\" (UID: \"857bc527-8cca-423d-b226-621d7fc41e37\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"
Sep 17 03:25:55.814692 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.814609    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmtcl\" (UniqueName: \"kubernetes.io/projected/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-kube-api-access-gmtcl\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg\" (UID: \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"
Sep 17 03:25:55.814692 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.814652    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg\" (UID: \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"
Sep 17 03:25:55.814796 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.814735    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg\" (UID: \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"
Sep 17 03:25:55.814909 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.814890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg\" (UID: \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"
Sep 17 03:25:55.824719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.824695    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmtcl\" (UniqueName: \"kubernetes.io/projected/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-kube-api-access-gmtcl\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg\" (UID: \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"
Sep 17 03:25:55.865591 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.865546    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"]
Sep 17 03:25:55.868279 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.868252    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"
Sep 17 03:25:55.877161 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.877133    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"
Sep 17 03:25:55.878142 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.878117    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"]
Sep 17 03:25:55.915202 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.915164    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlxrc\" (UniqueName: \"kubernetes.io/projected/562dc6aa-6e46-4346-b2b9-5c622a7940a5-kube-api-access-mlxrc\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5\" (UID: \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"
Sep 17 03:25:55.915367 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.915231    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/562dc6aa-6e46-4346-b2b9-5c622a7940a5-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5\" (UID: \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"
Sep 17 03:25:55.915367 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.915268    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/562dc6aa-6e46-4346-b2b9-5c622a7940a5-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5\" (UID: \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"
Sep 17 03:25:55.915367 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.915294    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/857bc527-8cca-423d-b226-621d7fc41e37-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr\" (UID: \"857bc527-8cca-423d-b226-621d7fc41e37\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"
Sep 17 03:25:55.915367 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.915321    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/857bc527-8cca-423d-b226-621d7fc41e37-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr\" (UID: \"857bc527-8cca-423d-b226-621d7fc41e37\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"
Sep 17 03:25:55.915367 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.915341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5b9k\" (UniqueName: \"kubernetes.io/projected/857bc527-8cca-423d-b226-621d7fc41e37-kube-api-access-d5b9k\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr\" (UID: \"857bc527-8cca-423d-b226-621d7fc41e37\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"
Sep 17 03:25:55.915766 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.915741    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/857bc527-8cca-423d-b226-621d7fc41e37-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr\" (UID: \"857bc527-8cca-423d-b226-621d7fc41e37\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"
Sep 17 03:25:55.915813 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.915750    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/857bc527-8cca-423d-b226-621d7fc41e37-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr\" (UID: \"857bc527-8cca-423d-b226-621d7fc41e37\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"
Sep 17 03:25:55.928603 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.928516    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5b9k\" (UniqueName: \"kubernetes.io/projected/857bc527-8cca-423d-b226-621d7fc41e37-kube-api-access-d5b9k\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr\" (UID: \"857bc527-8cca-423d-b226-621d7fc41e37\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"
Sep 17 03:25:55.976591 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:55.976538    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"
Sep 17 03:25:56.016498 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.016459    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/562dc6aa-6e46-4346-b2b9-5c622a7940a5-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5\" (UID: \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"
Sep 17 03:25:56.016498 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.016501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/562dc6aa-6e46-4346-b2b9-5c622a7940a5-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5\" (UID: \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"
Sep 17 03:25:56.016719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.016540    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j9lm\" (UniqueName: \"kubernetes.io/projected/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-kube-api-access-4j9lm\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb\" (UID: \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"
Sep 17 03:25:56.016719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.016614    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb\" (UID: \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"
Sep 17 03:25:56.016719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.016671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlxrc\" (UniqueName: \"kubernetes.io/projected/562dc6aa-6e46-4346-b2b9-5c622a7940a5-kube-api-access-mlxrc\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5\" (UID: \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"
Sep 17 03:25:56.016719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.016692    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb\" (UID: \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"
Sep 17 03:25:56.016914 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.016896    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/562dc6aa-6e46-4346-b2b9-5c622a7940a5-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5\" (UID: \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"
Sep 17 03:25:56.016997 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.016972    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/562dc6aa-6e46-4346-b2b9-5c622a7940a5-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5\" (UID: \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"
Sep 17 03:25:56.026380 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.026315    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlxrc\" (UniqueName: \"kubernetes.io/projected/562dc6aa-6e46-4346-b2b9-5c622a7940a5-kube-api-access-mlxrc\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5\" (UID: \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"
Sep 17 03:25:56.078911 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.078824    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"
Sep 17 03:25:56.118118 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.118079    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb\" (UID: \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"
Sep 17 03:25:56.118312 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.118132    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb\" (UID: \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"
Sep 17 03:25:56.118312 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.118181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j9lm\" (UniqueName: \"kubernetes.io/projected/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-kube-api-access-4j9lm\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb\" (UID: \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"
Sep 17 03:25:56.118505 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.118483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb\" (UID: \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"
Sep 17 03:25:56.118577 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.118531    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb\" (UID: \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"
Sep 17 03:25:56.127714 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.127509    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j9lm\" (UniqueName: \"kubernetes.io/projected/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-kube-api-access-4j9lm\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb\" (UID: \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"
Sep 17 03:25:56.152752 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.152726    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"]
Sep 17 03:25:56.155190 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:25:56.155162    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec2df44_e4d1_4bf5_9bdf_f7df52243c60.slice/crio-f216d04b28246fa5f4047ea7ef7f34c014ddad80d7bc69576dd8fc4443e6f94e WatchSource:0}: Error finding container f216d04b28246fa5f4047ea7ef7f34c014ddad80d7bc69576dd8fc4443e6f94e: Status 404 returned error can't find the container with id f216d04b28246fa5f4047ea7ef7f34c014ddad80d7bc69576dd8fc4443e6f94e
Sep 17 03:25:56.179000 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.178974    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"
Sep 17 03:25:56.251360 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.251325    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"]
Sep 17 03:25:56.255190 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:25:56.255159    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod857bc527_8cca_423d_b226_621d7fc41e37.slice/crio-2b3e28e372eac0ce3f90afd221a8a6fbd8bd60d502a6aff41e5693bc169f3ea7 WatchSource:0}: Error finding container 2b3e28e372eac0ce3f90afd221a8a6fbd8bd60d502a6aff41e5693bc169f3ea7: Status 404 returned error can't find the container with id 2b3e28e372eac0ce3f90afd221a8a6fbd8bd60d502a6aff41e5693bc169f3ea7
Sep 17 03:25:56.349612 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.349586    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"]
Sep 17 03:25:56.351151 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:25:56.351124    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod562dc6aa_6e46_4346_b2b9_5c622a7940a5.slice/crio-dfde5bfa3f41bd32211bc64d07a545d826bd1044ef529df15fad065c9af0e6c8 WatchSource:0}: Error finding container dfde5bfa3f41bd32211bc64d07a545d826bd1044ef529df15fad065c9af0e6c8: Status 404 returned error can't find the container with id dfde5bfa3f41bd32211bc64d07a545d826bd1044ef529df15fad065c9af0e6c8
Sep 17 03:25:56.452296 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:56.452269    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"]
Sep 17 03:25:56.510405 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:25:56.510360    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda806d966_ce7e_44f0_9ab1_9f7826fa97f8.slice/crio-8ac49dd7d47268e6ab1d2d2a63409d9145df26b534595bc5ee2a8e1608f32fb1 WatchSource:0}: Error finding container 8ac49dd7d47268e6ab1d2d2a63409d9145df26b534595bc5ee2a8e1608f32fb1: Status 404 returned error can't find the container with id 8ac49dd7d47268e6ab1d2d2a63409d9145df26b534595bc5ee2a8e1608f32fb1
Sep 17 03:25:57.073401 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:57.073371    2576 generic.go:356] "Generic (PLEG): container finished" podID="8ec2df44-e4d1-4bf5-9bdf-f7df52243c60" containerID="aedb4f7981d3d66547cecfeb08cf0a9b6f3ab079ceff4c2fb32ffcd3c5c68cdd" exitCode=0
Sep 17 03:25:57.073930 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:57.073457    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg" event={"ID":"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60","Type":"ContainerDied","Data":"aedb4f7981d3d66547cecfeb08cf0a9b6f3ab079ceff4c2fb32ffcd3c5c68cdd"}
Sep 17 03:25:57.073930 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:57.073514    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg" event={"ID":"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60","Type":"ContainerStarted","Data":"f216d04b28246fa5f4047ea7ef7f34c014ddad80d7bc69576dd8fc4443e6f94e"}
Sep 17 03:25:57.075074 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:57.075053    2576 generic.go:356] "Generic (PLEG): container finished" podID="562dc6aa-6e46-4346-b2b9-5c622a7940a5" containerID="21594601b798485abc9d584bc0ed9734d04a3046f0d2d0df7775bb49a7bead2d" exitCode=0
Sep 17 03:25:57.075139 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:57.075123    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5" event={"ID":"562dc6aa-6e46-4346-b2b9-5c622a7940a5","Type":"ContainerDied","Data":"21594601b798485abc9d584bc0ed9734d04a3046f0d2d0df7775bb49a7bead2d"}
Sep 17 03:25:57.075693 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:57.075169    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5" event={"ID":"562dc6aa-6e46-4346-b2b9-5c622a7940a5","Type":"ContainerStarted","Data":"dfde5bfa3f41bd32211bc64d07a545d826bd1044ef529df15fad065c9af0e6c8"}
Sep 17 03:25:57.082136 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:57.082108    2576 generic.go:356] "Generic (PLEG): container finished" podID="a806d966-ce7e-44f0-9ab1-9f7826fa97f8" containerID="745b004d1cc5dbeab05f7b3f0da2642305fa664fdec95ce87928b1150b806a41" exitCode=0
Sep 17 03:25:57.082234 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:57.082184    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb" event={"ID":"a806d966-ce7e-44f0-9ab1-9f7826fa97f8","Type":"ContainerDied","Data":"745b004d1cc5dbeab05f7b3f0da2642305fa664fdec95ce87928b1150b806a41"}
Sep 17 03:25:57.082321 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:57.082302    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb" event={"ID":"a806d966-ce7e-44f0-9ab1-9f7826fa97f8","Type":"ContainerStarted","Data":"8ac49dd7d47268e6ab1d2d2a63409d9145df26b534595bc5ee2a8e1608f32fb1"}
Sep 17 03:25:57.083901 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:57.083877    2576 generic.go:356] "Generic (PLEG): container finished" podID="857bc527-8cca-423d-b226-621d7fc41e37" containerID="4f20df721066964945ff3c8c793546596598b8573cb06269d20c869e6b92fa0d" exitCode=0
Sep 17 03:25:57.083990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:57.083911    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr" event={"ID":"857bc527-8cca-423d-b226-621d7fc41e37","Type":"ContainerDied","Data":"4f20df721066964945ff3c8c793546596598b8573cb06269d20c869e6b92fa0d"}
Sep 17 03:25:57.083990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:57.083944    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr" event={"ID":"857bc527-8cca-423d-b226-621d7fc41e37","Type":"ContainerStarted","Data":"2b3e28e372eac0ce3f90afd221a8a6fbd8bd60d502a6aff41e5693bc169f3ea7"}
Sep 17 03:25:58.093043 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:58.093009    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg" event={"ID":"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60","Type":"ContainerStarted","Data":"589b54d362179e3bfbf123fe41f61c51322fee2a5b7eb485b05676430e9d3dbb"}
Sep 17 03:25:58.095575 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:58.095524    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5" event={"ID":"562dc6aa-6e46-4346-b2b9-5c622a7940a5","Type":"ContainerStarted","Data":"383508e7c0ed6c060448ed1ce6fddc7a373f5f77f12a6b04e58509d7870fbc82"}
Sep 17 03:25:58.098182 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:58.098144    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb" event={"ID":"a806d966-ce7e-44f0-9ab1-9f7826fa97f8","Type":"ContainerStarted","Data":"aeb421d5dcb411d07945a2447c9106d7e666da6736aa6bb7525d9c402c53828d"}
Sep 17 03:25:58.100221 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:58.100199    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr" event={"ID":"857bc527-8cca-423d-b226-621d7fc41e37","Type":"ContainerStarted","Data":"b2f9eb77e48b224a367c10301b8c80362775de11fb81e9162be2832d3c46dc39"}
Sep 17 03:25:59.106912 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:59.106881    2576 generic.go:356] "Generic (PLEG): container finished" podID="8ec2df44-e4d1-4bf5-9bdf-f7df52243c60" containerID="589b54d362179e3bfbf123fe41f61c51322fee2a5b7eb485b05676430e9d3dbb" exitCode=0
Sep 17 03:25:59.107358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:59.106954    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg" event={"ID":"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60","Type":"ContainerDied","Data":"589b54d362179e3bfbf123fe41f61c51322fee2a5b7eb485b05676430e9d3dbb"}
Sep 17 03:25:59.108875 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:59.108851    2576 generic.go:356] "Generic (PLEG): container finished" podID="562dc6aa-6e46-4346-b2b9-5c622a7940a5" containerID="383508e7c0ed6c060448ed1ce6fddc7a373f5f77f12a6b04e58509d7870fbc82" exitCode=0
Sep 17 03:25:59.108949 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:59.108897    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5" event={"ID":"562dc6aa-6e46-4346-b2b9-5c622a7940a5","Type":"ContainerDied","Data":"383508e7c0ed6c060448ed1ce6fddc7a373f5f77f12a6b04e58509d7870fbc82"}
Sep 17 03:25:59.111246 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:59.111221    2576 generic.go:356] "Generic (PLEG): container finished" podID="a806d966-ce7e-44f0-9ab1-9f7826fa97f8" containerID="aeb421d5dcb411d07945a2447c9106d7e666da6736aa6bb7525d9c402c53828d" exitCode=0
Sep 17 03:25:59.111416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:59.111308    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb" event={"ID":"a806d966-ce7e-44f0-9ab1-9f7826fa97f8","Type":"ContainerDied","Data":"aeb421d5dcb411d07945a2447c9106d7e666da6736aa6bb7525d9c402c53828d"}
Sep 17 03:25:59.113668 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:59.113642    2576 generic.go:356] "Generic (PLEG): container finished" podID="857bc527-8cca-423d-b226-621d7fc41e37" containerID="b2f9eb77e48b224a367c10301b8c80362775de11fb81e9162be2832d3c46dc39" exitCode=0
Sep 17 03:25:59.113668 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:59.113669    2576 generic.go:356] "Generic (PLEG): container finished" podID="857bc527-8cca-423d-b226-621d7fc41e37" containerID="6602bd52555f196c78b68337faf10361955be73c7bd9b07ee3a2f50f82750fc8" exitCode=0
Sep 17 03:25:59.113810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:59.113716    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr" event={"ID":"857bc527-8cca-423d-b226-621d7fc41e37","Type":"ContainerDied","Data":"b2f9eb77e48b224a367c10301b8c80362775de11fb81e9162be2832d3c46dc39"}
Sep 17 03:25:59.113810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:25:59.113757    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr" event={"ID":"857bc527-8cca-423d-b226-621d7fc41e37","Type":"ContainerDied","Data":"6602bd52555f196c78b68337faf10361955be73c7bd9b07ee3a2f50f82750fc8"}
Sep 17 03:26:00.121268 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.121231    2576 generic.go:356] "Generic (PLEG): container finished" podID="8ec2df44-e4d1-4bf5-9bdf-f7df52243c60" containerID="31e433044a77c4da5f87e1d5d3003bf6cfc0add4cc9669f8928eacb8414692dd" exitCode=0
Sep 17 03:26:00.121683 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.121325    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg" event={"ID":"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60","Type":"ContainerDied","Data":"31e433044a77c4da5f87e1d5d3003bf6cfc0add4cc9669f8928eacb8414692dd"}
Sep 17 03:26:00.123519 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.123491    2576 generic.go:356] "Generic (PLEG): container finished" podID="562dc6aa-6e46-4346-b2b9-5c622a7940a5" containerID="01b3b1c600be751bff2295a93a54ca5374f1b45d5f06841d69f80ff7a50a25ec" exitCode=0
Sep 17 03:26:00.123649 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.123576    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5" event={"ID":"562dc6aa-6e46-4346-b2b9-5c622a7940a5","Type":"ContainerDied","Data":"01b3b1c600be751bff2295a93a54ca5374f1b45d5f06841d69f80ff7a50a25ec"}
Sep 17 03:26:00.125754 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.125731    2576 generic.go:356] "Generic (PLEG): container finished" podID="a806d966-ce7e-44f0-9ab1-9f7826fa97f8" containerID="a1090480d530410765ca99779defa93b13099054b8f8ec261b9d037060cb01b3" exitCode=0
Sep 17 03:26:00.125877 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.125787    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb" event={"ID":"a806d966-ce7e-44f0-9ab1-9f7826fa97f8","Type":"ContainerDied","Data":"a1090480d530410765ca99779defa93b13099054b8f8ec261b9d037060cb01b3"}
Sep 17 03:26:00.247954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.247898    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"
Sep 17 03:26:00.354916 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.354876    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5b9k\" (UniqueName: \"kubernetes.io/projected/857bc527-8cca-423d-b226-621d7fc41e37-kube-api-access-d5b9k\") pod \"857bc527-8cca-423d-b226-621d7fc41e37\" (UID: \"857bc527-8cca-423d-b226-621d7fc41e37\") "
Sep 17 03:26:00.355104 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.355004    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/857bc527-8cca-423d-b226-621d7fc41e37-bundle\") pod \"857bc527-8cca-423d-b226-621d7fc41e37\" (UID: \"857bc527-8cca-423d-b226-621d7fc41e37\") "
Sep 17 03:26:00.355104 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.355026    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/857bc527-8cca-423d-b226-621d7fc41e37-util\") pod \"857bc527-8cca-423d-b226-621d7fc41e37\" (UID: \"857bc527-8cca-423d-b226-621d7fc41e37\") "
Sep 17 03:26:00.355724 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.355676    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857bc527-8cca-423d-b226-621d7fc41e37-bundle" (OuterVolumeSpecName: "bundle") pod "857bc527-8cca-423d-b226-621d7fc41e37" (UID: "857bc527-8cca-423d-b226-621d7fc41e37"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:26:00.357052 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.357019    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/857bc527-8cca-423d-b226-621d7fc41e37-kube-api-access-d5b9k" (OuterVolumeSpecName: "kube-api-access-d5b9k") pod "857bc527-8cca-423d-b226-621d7fc41e37" (UID: "857bc527-8cca-423d-b226-621d7fc41e37"). InnerVolumeSpecName "kube-api-access-d5b9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:26:00.360608 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.360587    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857bc527-8cca-423d-b226-621d7fc41e37-util" (OuterVolumeSpecName: "util") pod "857bc527-8cca-423d-b226-621d7fc41e37" (UID: "857bc527-8cca-423d-b226-621d7fc41e37"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:26:00.455874 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.455819    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/857bc527-8cca-423d-b226-621d7fc41e37-bundle\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:00.455874 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.455866    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/857bc527-8cca-423d-b226-621d7fc41e37-util\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:00.455874 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:00.455877    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5b9k\" (UniqueName: \"kubernetes.io/projected/857bc527-8cca-423d-b226-621d7fc41e37-kube-api-access-d5b9k\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:01.132682 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.132643    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr" event={"ID":"857bc527-8cca-423d-b226-621d7fc41e37","Type":"ContainerDied","Data":"2b3e28e372eac0ce3f90afd221a8a6fbd8bd60d502a6aff41e5693bc169f3ea7"}
Sep 17 03:26:01.133052 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.132703    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b3e28e372eac0ce3f90afd221a8a6fbd8bd60d502a6aff41e5693bc169f3ea7"
Sep 17 03:26:01.133052 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.132712    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2m5vr"
Sep 17 03:26:01.272681 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.272657    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"
Sep 17 03:26:01.320891 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.320869    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"
Sep 17 03:26:01.323799 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.323776    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"
Sep 17 03:26:01.362305 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.362278    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlxrc\" (UniqueName: \"kubernetes.io/projected/562dc6aa-6e46-4346-b2b9-5c622a7940a5-kube-api-access-mlxrc\") pod \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\" (UID: \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\") "
Sep 17 03:26:01.362498 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.362322    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/562dc6aa-6e46-4346-b2b9-5c622a7940a5-util\") pod \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\" (UID: \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\") "
Sep 17 03:26:01.362498 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.362390    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/562dc6aa-6e46-4346-b2b9-5c622a7940a5-bundle\") pod \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\" (UID: \"562dc6aa-6e46-4346-b2b9-5c622a7940a5\") "
Sep 17 03:26:01.363143 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.363082    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562dc6aa-6e46-4346-b2b9-5c622a7940a5-bundle" (OuterVolumeSpecName: "bundle") pod "562dc6aa-6e46-4346-b2b9-5c622a7940a5" (UID: "562dc6aa-6e46-4346-b2b9-5c622a7940a5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:26:01.364545 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.364518    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/562dc6aa-6e46-4346-b2b9-5c622a7940a5-kube-api-access-mlxrc" (OuterVolumeSpecName: "kube-api-access-mlxrc") pod "562dc6aa-6e46-4346-b2b9-5c622a7940a5" (UID: "562dc6aa-6e46-4346-b2b9-5c622a7940a5"). InnerVolumeSpecName "kube-api-access-mlxrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:26:01.367796 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.367759    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/562dc6aa-6e46-4346-b2b9-5c622a7940a5-util" (OuterVolumeSpecName: "util") pod "562dc6aa-6e46-4346-b2b9-5c622a7940a5" (UID: "562dc6aa-6e46-4346-b2b9-5c622a7940a5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:26:01.463540 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.463446    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-util\") pod \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\" (UID: \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\") "
Sep 17 03:26:01.463540 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.463503    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4j9lm\" (UniqueName: \"kubernetes.io/projected/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-kube-api-access-4j9lm\") pod \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\" (UID: \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\") "
Sep 17 03:26:01.463540 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.463526    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-bundle\") pod \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\" (UID: \"a806d966-ce7e-44f0-9ab1-9f7826fa97f8\") "
Sep 17 03:26:01.463787 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.463560    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmtcl\" (UniqueName: \"kubernetes.io/projected/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-kube-api-access-gmtcl\") pod \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\" (UID: \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\") "
Sep 17 03:26:01.463787 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.463586    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-bundle\") pod \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\" (UID: \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\") "
Sep 17 03:26:01.463787 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.463605    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-util\") pod \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\" (UID: \"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60\") "
Sep 17 03:26:01.463787 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.463780    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/562dc6aa-6e46-4346-b2b9-5c622a7940a5-bundle\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:01.463991 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.463791    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlxrc\" (UniqueName: \"kubernetes.io/projected/562dc6aa-6e46-4346-b2b9-5c622a7940a5-kube-api-access-mlxrc\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:01.463991 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.463801    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/562dc6aa-6e46-4346-b2b9-5c622a7940a5-util\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:01.464288 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.464182    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-bundle" (OuterVolumeSpecName: "bundle") pod "a806d966-ce7e-44f0-9ab1-9f7826fa97f8" (UID: "a806d966-ce7e-44f0-9ab1-9f7826fa97f8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:26:01.464288 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.464246    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-bundle" (OuterVolumeSpecName: "bundle") pod "8ec2df44-e4d1-4bf5-9bdf-f7df52243c60" (UID: "8ec2df44-e4d1-4bf5-9bdf-f7df52243c60"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:26:01.466220 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.466193    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-kube-api-access-gmtcl" (OuterVolumeSpecName: "kube-api-access-gmtcl") pod "8ec2df44-e4d1-4bf5-9bdf-f7df52243c60" (UID: "8ec2df44-e4d1-4bf5-9bdf-f7df52243c60"). InnerVolumeSpecName "kube-api-access-gmtcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:26:01.466220 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.466210    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-kube-api-access-4j9lm" (OuterVolumeSpecName: "kube-api-access-4j9lm") pod "a806d966-ce7e-44f0-9ab1-9f7826fa97f8" (UID: "a806d966-ce7e-44f0-9ab1-9f7826fa97f8"). InnerVolumeSpecName "kube-api-access-4j9lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:26:01.469731 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.469711    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-util" (OuterVolumeSpecName: "util") pod "a806d966-ce7e-44f0-9ab1-9f7826fa97f8" (UID: "a806d966-ce7e-44f0-9ab1-9f7826fa97f8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:26:01.469801 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.469777    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-util" (OuterVolumeSpecName: "util") pod "8ec2df44-e4d1-4bf5-9bdf-f7df52243c60" (UID: "8ec2df44-e4d1-4bf5-9bdf-f7df52243c60"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:26:01.564658 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.564622    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-util\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:01.564658 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.564654    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4j9lm\" (UniqueName: \"kubernetes.io/projected/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-kube-api-access-4j9lm\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:01.564658 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.564665    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a806d966-ce7e-44f0-9ab1-9f7826fa97f8-bundle\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:01.564882 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.564676    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmtcl\" (UniqueName: \"kubernetes.io/projected/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-kube-api-access-gmtcl\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:01.564882 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.564684    2576 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-bundle\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:01.564882 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:01.564692    2576 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ec2df44-e4d1-4bf5-9bdf-f7df52243c60-util\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:02.139059 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:02.139022    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg" event={"ID":"8ec2df44-e4d1-4bf5-9bdf-f7df52243c60","Type":"ContainerDied","Data":"f216d04b28246fa5f4047ea7ef7f34c014ddad80d7bc69576dd8fc4443e6f94e"}
Sep 17 03:26:02.139494 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:02.139075    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f216d04b28246fa5f4047ea7ef7f34c014ddad80d7bc69576dd8fc4443e6f94e"
Sep 17 03:26:02.139494 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:02.139033    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fm8bg"
Sep 17 03:26:02.141065 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:02.141041    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5"
Sep 17 03:26:02.141065 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:02.141059    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847abj8n5" event={"ID":"562dc6aa-6e46-4346-b2b9-5c622a7940a5","Type":"ContainerDied","Data":"dfde5bfa3f41bd32211bc64d07a545d826bd1044ef529df15fad065c9af0e6c8"}
Sep 17 03:26:02.141232 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:02.141096    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dfde5bfa3f41bd32211bc64d07a545d826bd1044ef529df15fad065c9af0e6c8"
Sep 17 03:26:02.143168 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:02.143144    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb" event={"ID":"a806d966-ce7e-44f0-9ab1-9f7826fa97f8","Type":"ContainerDied","Data":"8ac49dd7d47268e6ab1d2d2a63409d9145df26b534595bc5ee2a8e1608f32fb1"}
Sep 17 03:26:02.143168 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:02.143164    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7nt5jb"
Sep 17 03:26:02.143307 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:02.143185    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ac49dd7d47268e6ab1d2d2a63409d9145df26b534595bc5ee2a8e1608f32fb1"
Sep 17 03:26:03.728704 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:03.728664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:04.499577 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:04.499546    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:06.983574 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983543    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-cbrdc"]
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983821    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a806d966-ce7e-44f0-9ab1-9f7826fa97f8" containerName="extract"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983832    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a806d966-ce7e-44f0-9ab1-9f7826fa97f8" containerName="extract"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983843    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a806d966-ce7e-44f0-9ab1-9f7826fa97f8" containerName="util"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983848    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a806d966-ce7e-44f0-9ab1-9f7826fa97f8" containerName="util"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983856    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="857bc527-8cca-423d-b226-621d7fc41e37" containerName="util"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983861    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="857bc527-8cca-423d-b226-621d7fc41e37" containerName="util"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983866    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562dc6aa-6e46-4346-b2b9-5c622a7940a5" containerName="util"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983871    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="562dc6aa-6e46-4346-b2b9-5c622a7940a5" containerName="util"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983881    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec2df44-e4d1-4bf5-9bdf-f7df52243c60" containerName="pull"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983887    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec2df44-e4d1-4bf5-9bdf-f7df52243c60" containerName="pull"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983893    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562dc6aa-6e46-4346-b2b9-5c622a7940a5" containerName="extract"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983898    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="562dc6aa-6e46-4346-b2b9-5c622a7940a5" containerName="extract"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983905    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec2df44-e4d1-4bf5-9bdf-f7df52243c60" containerName="extract"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983910    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec2df44-e4d1-4bf5-9bdf-f7df52243c60" containerName="extract"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983917    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ec2df44-e4d1-4bf5-9bdf-f7df52243c60" containerName="util"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983922    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ec2df44-e4d1-4bf5-9bdf-f7df52243c60" containerName="util"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983929    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="857bc527-8cca-423d-b226-621d7fc41e37" containerName="extract"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983935    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="857bc527-8cca-423d-b226-621d7fc41e37" containerName="extract"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983942    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562dc6aa-6e46-4346-b2b9-5c622a7940a5" containerName="pull"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983946    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="562dc6aa-6e46-4346-b2b9-5c622a7940a5" containerName="pull"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983953    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="857bc527-8cca-423d-b226-621d7fc41e37" containerName="pull"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983958    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="857bc527-8cca-423d-b226-621d7fc41e37" containerName="pull"
Sep 17 03:26:06.983954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983964    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a806d966-ce7e-44f0-9ab1-9f7826fa97f8" containerName="pull"
Sep 17 03:26:06.984620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.983970    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a806d966-ce7e-44f0-9ab1-9f7826fa97f8" containerName="pull"
Sep 17 03:26:06.984620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.984017    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="857bc527-8cca-423d-b226-621d7fc41e37" containerName="extract"
Sep 17 03:26:06.984620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.984025    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ec2df44-e4d1-4bf5-9bdf-f7df52243c60" containerName="extract"
Sep 17 03:26:06.984620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.984031    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="562dc6aa-6e46-4346-b2b9-5c622a7940a5" containerName="extract"
Sep 17 03:26:06.984620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.984036    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a806d966-ce7e-44f0-9ab1-9f7826fa97f8" containerName="extract"
Sep 17 03:26:06.986679 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.986659    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-cbrdc"
Sep 17 03:26:06.989414 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.989391    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 17 03:26:06.989575 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.989455    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 17 03:26:06.989575 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.989565    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8pmg2\""
Sep 17 03:26:06.996982 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:06.996960    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-cbrdc"]
Sep 17 03:26:07.111568 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:07.111532    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbnl5\" (UniqueName: \"kubernetes.io/projected/d41ae147-d2d2-4ac8-83c3-3d53ed056ca5-kube-api-access-lbnl5\") pod \"authorino-operator-76b478bd78-cbrdc\" (UID: \"d41ae147-d2d2-4ac8-83c3-3d53ed056ca5\") " pod="kuadrant-system/authorino-operator-76b478bd78-cbrdc"
Sep 17 03:26:07.212443 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:07.212389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbnl5\" (UniqueName: \"kubernetes.io/projected/d41ae147-d2d2-4ac8-83c3-3d53ed056ca5-kube-api-access-lbnl5\") pod \"authorino-operator-76b478bd78-cbrdc\" (UID: \"d41ae147-d2d2-4ac8-83c3-3d53ed056ca5\") " pod="kuadrant-system/authorino-operator-76b478bd78-cbrdc"
Sep 17 03:26:07.225876 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:07.225854    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbnl5\" (UniqueName: \"kubernetes.io/projected/d41ae147-d2d2-4ac8-83c3-3d53ed056ca5-kube-api-access-lbnl5\") pod \"authorino-operator-76b478bd78-cbrdc\" (UID: \"d41ae147-d2d2-4ac8-83c3-3d53ed056ca5\") " pod="kuadrant-system/authorino-operator-76b478bd78-cbrdc"
Sep 17 03:26:07.297510 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:07.297417    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-cbrdc"
Sep 17 03:26:07.575172 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:07.575147    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-cbrdc"]
Sep 17 03:26:07.577150 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:26:07.577123    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41ae147_d2d2_4ac8_83c3_3d53ed056ca5.slice/crio-45e81bc854917d72cd092f42595dc71debb48cd49bbc9c7b106b2681d016500e WatchSource:0}: Error finding container 45e81bc854917d72cd092f42595dc71debb48cd49bbc9c7b106b2681d016500e: Status 404 returned error can't find the container with id 45e81bc854917d72cd092f42595dc71debb48cd49bbc9c7b106b2681d016500e
Sep 17 03:26:07.715688 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:07.715651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:07.975743 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:07.975640    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:08.172464 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:08.172415    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-cbrdc" event={"ID":"d41ae147-d2d2-4ac8-83c3-3d53ed056ca5","Type":"ContainerStarted","Data":"45e81bc854917d72cd092f42595dc71debb48cd49bbc9c7b106b2681d016500e"}
Sep 17 03:26:09.697878 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:09.697840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:09.971966 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:09.971938    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:10.118447 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:10.118410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:10.186864 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:10.186815    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-cbrdc" event={"ID":"d41ae147-d2d2-4ac8-83c3-3d53ed056ca5","Type":"ContainerStarted","Data":"b496710dd2f11bbc9851b8d4057981dbd96410a7335f49f53aa182e7e689e8f7"}
Sep 17 03:26:10.187192 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:10.187166    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-cbrdc"
Sep 17 03:26:10.204830 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:10.204774    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-cbrdc" podStartSLOduration=1.825124296 podStartE2EDuration="4.204758851s" totalImagesPullingTime="2.379634555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:26:10.203561449 +0000 UTC m=+398.548287529" watchObservedRunningTime="2026-09-17 03:26:10.204758851 +0000 UTC m=+398.549484930"
Sep 17 03:26:10.735348 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:10.735308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:11.049680 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.049593    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:11.150919 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.150888    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:11.669404 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.669370    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f94cc7445-h87bm"]
Sep 17 03:26:11.671584 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.671569    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.674457 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.674409    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 03:26:11.675025 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.675008    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 03:26:11.675099 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.675009    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 03:26:11.675099 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.675009    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 03:26:11.675384 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.675360    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 03:26:11.675518 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.675363    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 03:26:11.675518 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.675363    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5fg2h\""
Sep 17 03:26:11.675518 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.675410    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 03:26:11.681789 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.681770    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 03:26:11.689265 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.689243    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f94cc7445-h87bm"]
Sep 17 03:26:11.748563 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.748525    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:11.843961 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.843925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:11.850068 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.850045    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4f6ef90-b889-467e-afa2-fd95f0f0087d-console-oauth-config\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.850155 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.850079    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t5dt\" (UniqueName: \"kubernetes.io/projected/f4f6ef90-b889-467e-afa2-fd95f0f0087d-kube-api-access-6t5dt\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.850155 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.850131    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f6ef90-b889-467e-afa2-fd95f0f0087d-service-ca\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.850255 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.850155    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4f6ef90-b889-467e-afa2-fd95f0f0087d-console-config\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.850255 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.850173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4f6ef90-b889-467e-afa2-fd95f0f0087d-oauth-serving-cert\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.850255 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.850240    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4f6ef90-b889-467e-afa2-fd95f0f0087d-console-serving-cert\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.850370 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.850316    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4f6ef90-b889-467e-afa2-fd95f0f0087d-trusted-ca-bundle\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.948598 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.948512    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:11.951647 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.951626    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f6ef90-b889-467e-afa2-fd95f0f0087d-service-ca\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.951714 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.951656    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4f6ef90-b889-467e-afa2-fd95f0f0087d-console-config\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.951714 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.951674    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4f6ef90-b889-467e-afa2-fd95f0f0087d-oauth-serving-cert\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.951832 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.951744    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4f6ef90-b889-467e-afa2-fd95f0f0087d-console-serving-cert\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.951832 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.951788    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4f6ef90-b889-467e-afa2-fd95f0f0087d-trusted-ca-bundle\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.951933 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.951833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4f6ef90-b889-467e-afa2-fd95f0f0087d-console-oauth-config\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.952073 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.952047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t5dt\" (UniqueName: \"kubernetes.io/projected/f4f6ef90-b889-467e-afa2-fd95f0f0087d-kube-api-access-6t5dt\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.952586 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.952561    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4f6ef90-b889-467e-afa2-fd95f0f0087d-service-ca\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.952683 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.952657    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4f6ef90-b889-467e-afa2-fd95f0f0087d-oauth-serving-cert\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.952864 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.952838    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4f6ef90-b889-467e-afa2-fd95f0f0087d-console-config\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.952922 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.952866    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4f6ef90-b889-467e-afa2-fd95f0f0087d-trusted-ca-bundle\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.954283 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.954264    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4f6ef90-b889-467e-afa2-fd95f0f0087d-console-serving-cert\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.954373 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.954318    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4f6ef90-b889-467e-afa2-fd95f0f0087d-console-oauth-config\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.960483 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.960463    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t5dt\" (UniqueName: \"kubernetes.io/projected/f4f6ef90-b889-467e-afa2-fd95f0f0087d-kube-api-access-6t5dt\") pod \"console-f94cc7445-h87bm\" (UID: \"f4f6ef90-b889-467e-afa2-fd95f0f0087d\") " pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:11.981403 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:11.981376    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:12.255321 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:12.255298    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f94cc7445-h87bm"]
Sep 17 03:26:12.256915 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:26:12.256884    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f6ef90_b889_467e_afa2_fd95f0f0087d.slice/crio-f150d8c6f896f93051da289faf2078647a970587bb3492604580b08d5a32fd23 WatchSource:0}: Error finding container f150d8c6f896f93051da289faf2078647a970587bb3492604580b08d5a32fd23: Status 404 returned error can't find the container with id f150d8c6f896f93051da289faf2078647a970587bb3492604580b08d5a32fd23
Sep 17 03:26:12.472648 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:12.472605    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:12.583620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:12.583581    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:13.202570 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.202531    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f94cc7445-h87bm" event={"ID":"f4f6ef90-b889-467e-afa2-fd95f0f0087d","Type":"ContainerStarted","Data":"94a5db0f14e9fb3ded01b88f963552a30527e2c5659a69205143b386bd688ee0"}
Sep 17 03:26:13.202948 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.202604    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f94cc7445-h87bm" event={"ID":"f4f6ef90-b889-467e-afa2-fd95f0f0087d","Type":"ContainerStarted","Data":"f150d8c6f896f93051da289faf2078647a970587bb3492604580b08d5a32fd23"}
Sep 17 03:26:13.226314 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.226263    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f94cc7445-h87bm" podStartSLOduration=2.226246901 podStartE2EDuration="2.226246901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:26:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:26:13.225377879 +0000 UTC m=+401.570103959" watchObservedRunningTime="2026-09-17 03:26:13.226246901 +0000 UTC m=+401.570972980"
Sep 17 03:26:13.666977 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.666938    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"]
Sep 17 03:26:13.669156 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.669129    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:13.671197 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.671180    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2tz2l\""
Sep 17 03:26:13.686561 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.686535    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"]
Sep 17 03:26:13.770106 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.770070    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e00d4375-0bdd-43c2-a189-42e00359d830-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5mh5v\" (UID: \"e00d4375-0bdd-43c2-a189-42e00359d830\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:13.770257 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.770114    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd8wf\" (UniqueName: \"kubernetes.io/projected/e00d4375-0bdd-43c2-a189-42e00359d830-kube-api-access-nd8wf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5mh5v\" (UID: \"e00d4375-0bdd-43c2-a189-42e00359d830\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:13.870848 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.870802    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e00d4375-0bdd-43c2-a189-42e00359d830-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5mh5v\" (UID: \"e00d4375-0bdd-43c2-a189-42e00359d830\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:13.870848 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.870850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd8wf\" (UniqueName: \"kubernetes.io/projected/e00d4375-0bdd-43c2-a189-42e00359d830-kube-api-access-nd8wf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5mh5v\" (UID: \"e00d4375-0bdd-43c2-a189-42e00359d830\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:13.871200 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.871178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e00d4375-0bdd-43c2-a189-42e00359d830-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5mh5v\" (UID: \"e00d4375-0bdd-43c2-a189-42e00359d830\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:13.878905 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.878876    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd8wf\" (UniqueName: \"kubernetes.io/projected/e00d4375-0bdd-43c2-a189-42e00359d830-kube-api-access-nd8wf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-5mh5v\" (UID: \"e00d4375-0bdd-43c2-a189-42e00359d830\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:13.979074 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:13.978985    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:14.115262 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:14.115232    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:14.254510 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:14.254484    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"]
Sep 17 03:26:14.256969 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:26:14.256938    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00d4375_0bdd_43c2_a189_42e00359d830.slice/crio-0ad7072a5c88fa25c0029b9718964768a1f38163b3b51503e67718dc9884935e WatchSource:0}: Error finding container 0ad7072a5c88fa25c0029b9718964768a1f38163b3b51503e67718dc9884935e: Status 404 returned error can't find the container with id 0ad7072a5c88fa25c0029b9718964768a1f38163b3b51503e67718dc9884935e
Sep 17 03:26:15.131534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:15.131497    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:15.217183 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:15.217140    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v" event={"ID":"e00d4375-0bdd-43c2-a189-42e00359d830","Type":"ContainerStarted","Data":"0ad7072a5c88fa25c0029b9718964768a1f38163b3b51503e67718dc9884935e"}
Sep 17 03:26:17.138170 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:17.138135    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:17.249273 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:17.249236    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:17.512665 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:17.512569    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:17.617924 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:17.617890    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:18.236750 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:18.236657    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v" event={"ID":"e00d4375-0bdd-43c2-a189-42e00359d830","Type":"ContainerStarted","Data":"b8b60312c1c154cd965f04278927d8051ec0b120f9ca3c09cae74be2734f94ad"}
Sep 17 03:26:18.237170 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:18.236752    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:18.263078 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:18.263018    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v" podStartSLOduration=1.536857358 podStartE2EDuration="5.262998846s" totalImagesPullingTime="3.726141488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:26:18.259390334 +0000 UTC m=+406.604116413" watchObservedRunningTime="2026-09-17 03:26:18.262998846 +0000 UTC m=+406.607724924"
Sep 17 03:26:20.636134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:20.636090    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:20.825744 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:20.825708    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:20.927421 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:20.927313    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:21.027004 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:21.026970    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:21.194654 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:21.194564    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-cbrdc"
Sep 17 03:26:21.981828 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:21.981794    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:21.982227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:21.981856    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:21.986577 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:21.986548    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:22.264839 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:22.264763    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f94cc7445-h87bm"
Sep 17 03:26:22.521100 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:22.521015    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:22.703378 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:22.703342    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:22.802831 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:22.802793    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:24.813145 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:24.813109    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:24.913032 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:24.912998    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:25.010880 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:25.010833    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:29.244246 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:29.244211    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:31.324201 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.324143    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:31.426184 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.426147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:31.570815 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.570772    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"]
Sep 17 03:26:31.571059 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.571033    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v" podUID="e00d4375-0bdd-43c2-a189-42e00359d830" containerName="manager" containerID="cri-o://b8b60312c1c154cd965f04278927d8051ec0b120f9ca3c09cae74be2734f94ad" gracePeriod=2
Sep 17 03:26:31.577866 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.577791    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"]
Sep 17 03:26:31.617946 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.617912    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"]
Sep 17 03:26:31.618349 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.618330    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e00d4375-0bdd-43c2-a189-42e00359d830" containerName="manager"
Sep 17 03:26:31.618424 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.618351    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e00d4375-0bdd-43c2-a189-42e00359d830" containerName="manager"
Sep 17 03:26:31.618508 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.618470    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e00d4375-0bdd-43c2-a189-42e00359d830" containerName="manager"
Sep 17 03:26:31.620498 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.620477    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"
Sep 17 03:26:31.662315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.662276    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"]
Sep 17 03:26:31.696821 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:26:31.696782    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-5mh5v\" is forbidden: User \"system:node:ip-10-0-137-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-141.ec2.internal' and this object" podUID="e00d4375-0bdd-43c2-a189-42e00359d830" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:31.709192 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.709164    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-276mx\" (UniqueName: \"kubernetes.io/projected/6366d116-54bf-4a47-b233-3c1a3bf401c4-kube-api-access-276mx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mnt7d\" (UID: \"6366d116-54bf-4a47-b233-3c1a3bf401c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"
Sep 17 03:26:31.709319 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.709196    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6366d116-54bf-4a47-b233-3c1a3bf401c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mnt7d\" (UID: \"6366d116-54bf-4a47-b233-3c1a3bf401c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"
Sep 17 03:26:31.809660 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.809632    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:31.810000 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.809979    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-276mx\" (UniqueName: \"kubernetes.io/projected/6366d116-54bf-4a47-b233-3c1a3bf401c4-kube-api-access-276mx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mnt7d\" (UID: \"6366d116-54bf-4a47-b233-3c1a3bf401c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"
Sep 17 03:26:31.810043 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.810009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6366d116-54bf-4a47-b233-3c1a3bf401c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mnt7d\" (UID: \"6366d116-54bf-4a47-b233-3c1a3bf401c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"
Sep 17 03:26:31.810300 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.810286    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6366d116-54bf-4a47-b233-3c1a3bf401c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mnt7d\" (UID: \"6366d116-54bf-4a47-b233-3c1a3bf401c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"
Sep 17 03:26:31.811740 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:26:31.811717    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-5mh5v\" is forbidden: User \"system:node:ip-10-0-137-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-141.ec2.internal' and this object" podUID="e00d4375-0bdd-43c2-a189-42e00359d830" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:31.824050 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.824031    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-276mx\" (UniqueName: \"kubernetes.io/projected/6366d116-54bf-4a47-b233-3c1a3bf401c4-kube-api-access-276mx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-mnt7d\" (UID: \"6366d116-54bf-4a47-b233-3c1a3bf401c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"
Sep 17 03:26:31.910603 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.910505    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd8wf\" (UniqueName: \"kubernetes.io/projected/e00d4375-0bdd-43c2-a189-42e00359d830-kube-api-access-nd8wf\") pod \"e00d4375-0bdd-43c2-a189-42e00359d830\" (UID: \"e00d4375-0bdd-43c2-a189-42e00359d830\") "
Sep 17 03:26:31.910603 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.910605    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e00d4375-0bdd-43c2-a189-42e00359d830-extensions-socket-volume\") pod \"e00d4375-0bdd-43c2-a189-42e00359d830\" (UID: \"e00d4375-0bdd-43c2-a189-42e00359d830\") "
Sep 17 03:26:31.911068 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.911042    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00d4375-0bdd-43c2-a189-42e00359d830-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e00d4375-0bdd-43c2-a189-42e00359d830" (UID: "e00d4375-0bdd-43c2-a189-42e00359d830"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:26:31.912796 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.912770    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e00d4375-0bdd-43c2-a189-42e00359d830-kube-api-access-nd8wf" (OuterVolumeSpecName: "kube-api-access-nd8wf") pod "e00d4375-0bdd-43c2-a189-42e00359d830" (UID: "e00d4375-0bdd-43c2-a189-42e00359d830"). InnerVolumeSpecName "kube-api-access-nd8wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:26:31.948904 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:31.948877    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"
Sep 17 03:26:32.011958 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:32.011926    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e00d4375-0bdd-43c2-a189-42e00359d830-extensions-socket-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:32.011958 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:32.011955    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd8wf\" (UniqueName: \"kubernetes.io/projected/e00d4375-0bdd-43c2-a189-42e00359d830-kube-api-access-nd8wf\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:32.223042 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:32.223018    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"]
Sep 17 03:26:32.224677 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:26:32.224652    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6366d116_54bf_4a47_b233_3c1a3bf401c4.slice/crio-05b25997b9f62ca90b5362a6a8988c6e6c5501adb2149665ac069e7b00c174be WatchSource:0}: Error finding container 05b25997b9f62ca90b5362a6a8988c6e6c5501adb2149665ac069e7b00c174be: Status 404 returned error can't find the container with id 05b25997b9f62ca90b5362a6a8988c6e6c5501adb2149665ac069e7b00c174be
Sep 17 03:26:32.295681 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:26:32.295643    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-5mh5v\" is forbidden: User \"system:node:ip-10-0-137-141.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-141.ec2.internal' and this object" podUID="e00d4375-0bdd-43c2-a189-42e00359d830" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:32.297738 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:32.297709    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e00d4375-0bdd-43c2-a189-42e00359d830" path="/var/lib/kubelet/pods/e00d4375-0bdd-43c2-a189-42e00359d830/volumes"
Sep 17 03:26:32.311807 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:32.311779    2576 generic.go:356] "Generic (PLEG): container finished" podID="e00d4375-0bdd-43c2-a189-42e00359d830" containerID="b8b60312c1c154cd965f04278927d8051ec0b120f9ca3c09cae74be2734f94ad" exitCode=0
Sep 17 03:26:32.311915 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:32.311822    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-5mh5v"
Sep 17 03:26:32.311915 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:32.311876    2576 scope.go:117] "RemoveContainer" containerID="b8b60312c1c154cd965f04278927d8051ec0b120f9ca3c09cae74be2734f94ad"
Sep 17 03:26:32.313938 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:32.313913    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d" event={"ID":"6366d116-54bf-4a47-b233-3c1a3bf401c4","Type":"ContainerStarted","Data":"16fba3289fb004cafa45c1ed3a1e0479430289ce2f729a5723cf4b4f6e16b3d0"}
Sep 17 03:26:32.314047 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:32.313957    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d" event={"ID":"6366d116-54bf-4a47-b233-3c1a3bf401c4","Type":"ContainerStarted","Data":"05b25997b9f62ca90b5362a6a8988c6e6c5501adb2149665ac069e7b00c174be"}
Sep 17 03:26:32.314108 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:32.314092    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"
Sep 17 03:26:32.320329 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:32.320313    2576 scope.go:117] "RemoveContainer" containerID="b8b60312c1c154cd965f04278927d8051ec0b120f9ca3c09cae74be2734f94ad"
Sep 17 03:26:32.320620 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:26:32.320601    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8b60312c1c154cd965f04278927d8051ec0b120f9ca3c09cae74be2734f94ad\": container with ID starting with b8b60312c1c154cd965f04278927d8051ec0b120f9ca3c09cae74be2734f94ad not found: ID does not exist" containerID="b8b60312c1c154cd965f04278927d8051ec0b120f9ca3c09cae74be2734f94ad"
Sep 17 03:26:32.320686 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:32.320626    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8b60312c1c154cd965f04278927d8051ec0b120f9ca3c09cae74be2734f94ad"} err="failed to get container status \"b8b60312c1c154cd965f04278927d8051ec0b120f9ca3c09cae74be2734f94ad\": rpc error: code = NotFound desc = could not find container \"b8b60312c1c154cd965f04278927d8051ec0b120f9ca3c09cae74be2734f94ad\": container with ID starting with b8b60312c1c154cd965f04278927d8051ec0b120f9ca3c09cae74be2734f94ad not found: ID does not exist"
Sep 17 03:26:32.334730 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:32.334687    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d" podStartSLOduration=1.334674094 podStartE2EDuration="1.334674094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:26:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:26:32.332993785 +0000 UTC m=+420.677719864" watchObservedRunningTime="2026-09-17 03:26:32.334674094 +0000 UTC m=+420.679400172"
Sep 17 03:26:36.845643 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:36.845599    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:36.906155 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:36.906110    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:37.471582 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:37.471548    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:37.596066 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:37.596031    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:39.858507 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:39.858474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:39.959818 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:39.959778    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:43.322352 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:43.322319    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"
Sep 17 03:26:45.053199 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:45.053166    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:45.156604 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:45.156568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:45.253883 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:45.253848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:45.354451 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:45.354344    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:45.454457 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:45.454407    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:45.511956 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:45.511921    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:47.051356 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.051322    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"]
Sep 17 03:26:47.051777 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.051548    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d" podUID="6366d116-54bf-4a47-b233-3c1a3bf401c4" containerName="manager" containerID="cri-o://16fba3289fb004cafa45c1ed3a1e0479430289ce2f729a5723cf4b4f6e16b3d0" gracePeriod=10
Sep 17 03:26:47.299197 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.299170    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"
Sep 17 03:26:47.336070 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.335996    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-276mx\" (UniqueName: \"kubernetes.io/projected/6366d116-54bf-4a47-b233-3c1a3bf401c4-kube-api-access-276mx\") pod \"6366d116-54bf-4a47-b233-3c1a3bf401c4\" (UID: \"6366d116-54bf-4a47-b233-3c1a3bf401c4\") "
Sep 17 03:26:47.336070 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.336059    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6366d116-54bf-4a47-b233-3c1a3bf401c4-extensions-socket-volume\") pod \"6366d116-54bf-4a47-b233-3c1a3bf401c4\" (UID: \"6366d116-54bf-4a47-b233-3c1a3bf401c4\") "
Sep 17 03:26:47.336519 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.336491    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6366d116-54bf-4a47-b233-3c1a3bf401c4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6366d116-54bf-4a47-b233-3c1a3bf401c4" (UID: "6366d116-54bf-4a47-b233-3c1a3bf401c4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:26:47.338175 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.338148    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6366d116-54bf-4a47-b233-3c1a3bf401c4-kube-api-access-276mx" (OuterVolumeSpecName: "kube-api-access-276mx") pod "6366d116-54bf-4a47-b233-3c1a3bf401c4" (UID: "6366d116-54bf-4a47-b233-3c1a3bf401c4"). InnerVolumeSpecName "kube-api-access-276mx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:26:47.388511 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.388474    2576 generic.go:356] "Generic (PLEG): container finished" podID="6366d116-54bf-4a47-b233-3c1a3bf401c4" containerID="16fba3289fb004cafa45c1ed3a1e0479430289ce2f729a5723cf4b4f6e16b3d0" exitCode=0
Sep 17 03:26:47.388707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.388523    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"
Sep 17 03:26:47.388707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.388558    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d" event={"ID":"6366d116-54bf-4a47-b233-3c1a3bf401c4","Type":"ContainerDied","Data":"16fba3289fb004cafa45c1ed3a1e0479430289ce2f729a5723cf4b4f6e16b3d0"}
Sep 17 03:26:47.388707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.388613    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d" event={"ID":"6366d116-54bf-4a47-b233-3c1a3bf401c4","Type":"ContainerDied","Data":"05b25997b9f62ca90b5362a6a8988c6e6c5501adb2149665ac069e7b00c174be"}
Sep 17 03:26:47.388707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.388642    2576 scope.go:117] "RemoveContainer" containerID="16fba3289fb004cafa45c1ed3a1e0479430289ce2f729a5723cf4b4f6e16b3d0"
Sep 17 03:26:47.398279 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.398250    2576 scope.go:117] "RemoveContainer" containerID="16fba3289fb004cafa45c1ed3a1e0479430289ce2f729a5723cf4b4f6e16b3d0"
Sep 17 03:26:47.398623 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:26:47.398605    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16fba3289fb004cafa45c1ed3a1e0479430289ce2f729a5723cf4b4f6e16b3d0\": container with ID starting with 16fba3289fb004cafa45c1ed3a1e0479430289ce2f729a5723cf4b4f6e16b3d0 not found: ID does not exist" containerID="16fba3289fb004cafa45c1ed3a1e0479430289ce2f729a5723cf4b4f6e16b3d0"
Sep 17 03:26:47.398701 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.398630    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16fba3289fb004cafa45c1ed3a1e0479430289ce2f729a5723cf4b4f6e16b3d0"} err="failed to get container status \"16fba3289fb004cafa45c1ed3a1e0479430289ce2f729a5723cf4b4f6e16b3d0\": rpc error: code = NotFound desc = could not find container \"16fba3289fb004cafa45c1ed3a1e0479430289ce2f729a5723cf4b4f6e16b3d0\": container with ID starting with 16fba3289fb004cafa45c1ed3a1e0479430289ce2f729a5723cf4b4f6e16b3d0 not found: ID does not exist"
Sep 17 03:26:47.410029 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.409995    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"]
Sep 17 03:26:47.420226 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.420186    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-mnt7d"]
Sep 17 03:26:47.437462 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.437418    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-276mx\" (UniqueName: \"kubernetes.io/projected/6366d116-54bf-4a47-b233-3c1a3bf401c4-kube-api-access-276mx\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:47.437462 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.437454    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6366d116-54bf-4a47-b233-3c1a3bf401c4-extensions-socket-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:26:47.462621 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.462590    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"]
Sep 17 03:26:47.462888 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.462876    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6366d116-54bf-4a47-b233-3c1a3bf401c4" containerName="manager"
Sep 17 03:26:47.462888 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.462889    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6366d116-54bf-4a47-b233-3c1a3bf401c4" containerName="manager"
Sep 17 03:26:47.462981 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.462948    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6366d116-54bf-4a47-b233-3c1a3bf401c4" containerName="manager"
Sep 17 03:26:47.467694 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.467676    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:26:47.469869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.469849    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2tz2l\""
Sep 17 03:26:47.487669 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.487639    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"]
Sep 17 03:26:47.538834 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.538795    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktd5v\" (UniqueName: \"kubernetes.io/projected/b16dffd3-95d8-4fa6-ac56-fb30275e4c24-kube-api-access-ktd5v\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ctzbx\" (UID: \"b16dffd3-95d8-4fa6-ac56-fb30275e4c24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:26:47.539043 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.538927    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b16dffd3-95d8-4fa6-ac56-fb30275e4c24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ctzbx\" (UID: \"b16dffd3-95d8-4fa6-ac56-fb30275e4c24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:26:47.640340 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.640236    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b16dffd3-95d8-4fa6-ac56-fb30275e4c24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ctzbx\" (UID: \"b16dffd3-95d8-4fa6-ac56-fb30275e4c24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:26:47.640340 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.640298    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktd5v\" (UniqueName: \"kubernetes.io/projected/b16dffd3-95d8-4fa6-ac56-fb30275e4c24-kube-api-access-ktd5v\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ctzbx\" (UID: \"b16dffd3-95d8-4fa6-ac56-fb30275e4c24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:26:47.640679 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.640655    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b16dffd3-95d8-4fa6-ac56-fb30275e4c24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ctzbx\" (UID: \"b16dffd3-95d8-4fa6-ac56-fb30275e4c24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:26:47.649023 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.648999    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktd5v\" (UniqueName: \"kubernetes.io/projected/b16dffd3-95d8-4fa6-ac56-fb30275e4c24-kube-api-access-ktd5v\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ctzbx\" (UID: \"b16dffd3-95d8-4fa6-ac56-fb30275e4c24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:26:47.777726 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:47.777683    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:26:48.051299 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:48.051274    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"]
Sep 17 03:26:48.053260 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:26:48.053238    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb16dffd3_95d8_4fa6_ac56_fb30275e4c24.slice/crio-ebd7fd71451ee09e583eace7004eb18e7786187f3c185377bd1646e67e94cac8 WatchSource:0}: Error finding container ebd7fd71451ee09e583eace7004eb18e7786187f3c185377bd1646e67e94cac8: Status 404 returned error can't find the container with id ebd7fd71451ee09e583eace7004eb18e7786187f3c185377bd1646e67e94cac8
Sep 17 03:26:48.293881 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:48.293850    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6366d116-54bf-4a47-b233-3c1a3bf401c4" path="/var/lib/kubelet/pods/6366d116-54bf-4a47-b233-3c1a3bf401c4/volumes"
Sep 17 03:26:48.395650 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:48.395611    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx" event={"ID":"b16dffd3-95d8-4fa6-ac56-fb30275e4c24","Type":"ContainerStarted","Data":"59f827834ae9772fa54f3b9bad7d4ad30fb78d7760c2b082c84a3f9d658fe595"}
Sep 17 03:26:48.395837 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:48.395668    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx" event={"ID":"b16dffd3-95d8-4fa6-ac56-fb30275e4c24","Type":"ContainerStarted","Data":"ebd7fd71451ee09e583eace7004eb18e7786187f3c185377bd1646e67e94cac8"}
Sep 17 03:26:48.395837 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:48.395696    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:26:48.419241 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:48.419178    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx" podStartSLOduration=1.419160853 podStartE2EDuration="1.419160853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:26:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:26:48.417113309 +0000 UTC m=+436.761839390" watchObservedRunningTime="2026-09-17 03:26:48.419160853 +0000 UTC m=+436.763886932"
Sep 17 03:26:51.258958 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:51.258927    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:51.353911 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:51.353874    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:55.120938 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:55.120857    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:55.222315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:55.222280    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:26:59.403962 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:26:59.403928    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:27:03.460909 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:03.460850    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:03.723101 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:03.723007    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:17.060742 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.060703    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:17.159697 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.159663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:17.217545 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.217484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:17.382280 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.382207    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:27:17.385636 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.385613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-llwrp"
Sep 17 03:27:17.388005 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.387986    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 17 03:27:17.388111 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.387986    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f5jg8\""
Sep 17 03:27:17.394851 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.394829    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:27:17.424464 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.424409    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:27:17.493011 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.492972    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d2e653b6-974a-4bdd-b84e-09f1832ce26f-config-file\") pod \"limitador-limitador-84994f7ffc-llwrp\" (UID: \"d2e653b6-974a-4bdd-b84e-09f1832ce26f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-llwrp"
Sep 17 03:27:17.493181 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.493027    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dks6v\" (UniqueName: \"kubernetes.io/projected/d2e653b6-974a-4bdd-b84e-09f1832ce26f-kube-api-access-dks6v\") pod \"limitador-limitador-84994f7ffc-llwrp\" (UID: \"d2e653b6-974a-4bdd-b84e-09f1832ce26f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-llwrp"
Sep 17 03:27:17.594351 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.594314    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d2e653b6-974a-4bdd-b84e-09f1832ce26f-config-file\") pod \"limitador-limitador-84994f7ffc-llwrp\" (UID: \"d2e653b6-974a-4bdd-b84e-09f1832ce26f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-llwrp"
Sep 17 03:27:17.594579 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.594381    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dks6v\" (UniqueName: \"kubernetes.io/projected/d2e653b6-974a-4bdd-b84e-09f1832ce26f-kube-api-access-dks6v\") pod \"limitador-limitador-84994f7ffc-llwrp\" (UID: \"d2e653b6-974a-4bdd-b84e-09f1832ce26f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-llwrp"
Sep 17 03:27:17.595028 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.595004    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d2e653b6-974a-4bdd-b84e-09f1832ce26f-config-file\") pod \"limitador-limitador-84994f7ffc-llwrp\" (UID: \"d2e653b6-974a-4bdd-b84e-09f1832ce26f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-llwrp"
Sep 17 03:27:17.602479 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.602453    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dks6v\" (UniqueName: \"kubernetes.io/projected/d2e653b6-974a-4bdd-b84e-09f1832ce26f-kube-api-access-dks6v\") pod \"limitador-limitador-84994f7ffc-llwrp\" (UID: \"d2e653b6-974a-4bdd-b84e-09f1832ce26f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-llwrp"
Sep 17 03:27:17.696478 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.696356    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-llwrp"
Sep 17 03:27:17.878837 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.878801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:17.971554 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.971531    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:27:17.973513 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:27:17.973485    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e653b6_974a_4bdd_b84e_09f1832ce26f.slice/crio-0e4d2bc4a68add7382cf465a0f69f6f9977a0ec1ee3deca267f41cb6b2c2001e WatchSource:0}: Error finding container 0e4d2bc4a68add7382cf465a0f69f6f9977a0ec1ee3deca267f41cb6b2c2001e: Status 404 returned error can't find the container with id 0e4d2bc4a68add7382cf465a0f69f6f9977a0ec1ee3deca267f41cb6b2c2001e
Sep 17 03:27:17.976842 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:17.976819    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:18.554707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:18.554670    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-llwrp" event={"ID":"d2e653b6-974a-4bdd-b84e-09f1832ce26f","Type":"ContainerStarted","Data":"0e4d2bc4a68add7382cf465a0f69f6f9977a0ec1ee3deca267f41cb6b2c2001e"}
Sep 17 03:27:19.561727 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:19.561693    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-llwrp" event={"ID":"d2e653b6-974a-4bdd-b84e-09f1832ce26f","Type":"ContainerStarted","Data":"d240572b780b27bb973c4728e0a0637d4942b496eff05f68a3e57a0b93932bc1"}
Sep 17 03:27:19.562139 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:19.561811    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-llwrp"
Sep 17 03:27:19.588406 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:19.588349    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-llwrp" podStartSLOduration=1.589614562 podStartE2EDuration="2.588333782s" totalImagesPullingTime="998.71922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:27:19.586743726 +0000 UTC m=+467.931469806" watchObservedRunningTime="2026-09-17 03:27:19.588333782 +0000 UTC m=+467.933059861"
Sep 17 03:27:30.568212 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:30.568176    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-llwrp"
Sep 17 03:27:32.168666 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:32.168637    2576 scope.go:117] "RemoveContainer" containerID="782ee259cca0bb760e28364eed695fd7a87b3fdf8dae5933378f2382f3de6e0b"
Sep 17 03:27:32.440308 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:32.440228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:32.539228 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:32.539169    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:32.935589 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:32.935556    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:45.620044 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:45.620005    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:45.731829 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:45.731798    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:52.165025 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:52.164990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:52.239374 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:52.239330    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:52.297363 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:52.297324    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:52.534061 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:52.533968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:55.830855 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:55.830810    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:27:55.917073 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:27:55.917037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:28:05.810238 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:05.810203    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:28:05.872919 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:05.872882    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:28:30.426888 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:30.426781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:28:30.525240 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:30.525210    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:28:31.880991 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:31.880951    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:28:32.013357 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:32.013313    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:28:32.106263 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:32.106227    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:28:32.571067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:32.571037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:28:32.672304 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:32.672266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:28:32.810045 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:32.809995    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:28:42.701710 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:42.701674    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:28:42.831174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:42.831127    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:28:52.312819 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:52.312785    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:28:52.863462 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:52.863407    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:28:56.733516 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:28:56.733472    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:29:05.089100 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.089068    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"]
Sep 17 03:29:05.091977 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.091958    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.096026 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.096000    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 17 03:29:05.096026 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.096016    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 17 03:29:05.096221 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.096022    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 17 03:29:05.096221 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.096000    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-g74zs\""
Sep 17 03:29:05.106605 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.106577    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"]
Sep 17 03:29:05.116575 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.116546    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:29:05.124450 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.124407    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.124608 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.124455    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.124608 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.124478    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.124608 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.124563    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.124759 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.124606    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.124759 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.124640    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlvb7\" (UniqueName: \"kubernetes.io/projected/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-kube-api-access-qlvb7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.225739 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.225699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.225942 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.225752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.225942 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.225789    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.225942 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.225821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.225942 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.225846    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlvb7\" (UniqueName: \"kubernetes.io/projected/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-kube-api-access-qlvb7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.225942 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.225904    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.226304 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.226282    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.226362 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.226320    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.226362 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.226346    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.228166 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.228142    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.228442 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.228410    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.234736 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.234712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlvb7\" (UniqueName: \"kubernetes.io/projected/8175a1c8-b4a3-4a15-b5af-540e4cd1b645-kube-api-access-qlvb7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-q9b6v\" (UID: \"8175a1c8-b4a3-4a15-b5af-540e4cd1b645\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.403036 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.402950    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:05.676763 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.676738    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"]
Sep 17 03:29:05.679487 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:29:05.679454    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8175a1c8_b4a3_4a15_b5af_540e4cd1b645.slice/crio-6f4070eb12f27453f919fcfbcf37d53ae93a03893b64861b4ba076186ca7de99 WatchSource:0}: Error finding container 6f4070eb12f27453f919fcfbcf37d53ae93a03893b64861b4ba076186ca7de99: Status 404 returned error can't find the container with id 6f4070eb12f27453f919fcfbcf37d53ae93a03893b64861b4ba076186ca7de99
Sep 17 03:29:05.834923 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:05.834887    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:29:06.088314 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:06.088271    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v" event={"ID":"8175a1c8-b4a3-4a15-b5af-540e4cd1b645","Type":"ContainerStarted","Data":"6f4070eb12f27453f919fcfbcf37d53ae93a03893b64861b4ba076186ca7de99"}
Sep 17 03:29:07.096004 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:07.095966    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v" event={"ID":"8175a1c8-b4a3-4a15-b5af-540e4cd1b645","Type":"ContainerStarted","Data":"e753f2cd4d201388afb39924cf9fb104cd780e2ab28a0c7b0432cf1dc80aa2aa"}
Sep 17 03:29:07.113732 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:07.113679    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v" podStartSLOduration=1.154261144 podStartE2EDuration="2.113661322s" totalImagesPullingTime="959.400178ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:29:07.111088774 +0000 UTC m=+575.455814855" watchObservedRunningTime="2026-09-17 03:29:07.113661322 +0000 UTC m=+575.458387446"
Sep 17 03:29:09.830235 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:09.830203    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:29:09.931336 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:09.931298    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:29:15.129044 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:15.128996    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:29:15.403356 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:15.403266    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:15.403356 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:15.403329    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:15.416236 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:15.416209    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:16.153094 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:16.153066    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-q9b6v"
Sep 17 03:29:16.330116 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:16.330076    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:29:22.711838 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:22.711801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:29:24.141445 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:24.141409    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:29:32.186821 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:32.186790    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:29:32.187595 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:32.187572    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:29:34.318147 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:34.318111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:29:35.928891 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:35.928855    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:29:45.664318 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.664279    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"]
Sep 17 03:29:45.669305 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.669285    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.671514 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.671488    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 17 03:29:45.677458 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.677405    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"]
Sep 17 03:29:45.721473 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.721417    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:29:45.765627 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.765583    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fcc975f-291c-4f45-ba48-f9e54c161fdf-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.765822 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.765643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fcc975f-291c-4f45-ba48-f9e54c161fdf-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.765822 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.765699    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fcc975f-291c-4f45-ba48-f9e54c161fdf-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.765822 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.765784    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fcc975f-291c-4f45-ba48-f9e54c161fdf-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.765822 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.765812    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvp6c\" (UniqueName: \"kubernetes.io/projected/3fcc975f-291c-4f45-ba48-f9e54c161fdf-kube-api-access-rvp6c\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.765985 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.765849    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fcc975f-291c-4f45-ba48-f9e54c161fdf-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.866293 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.866253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fcc975f-291c-4f45-ba48-f9e54c161fdf-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.866293 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.866301    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fcc975f-291c-4f45-ba48-f9e54c161fdf-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.866657 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.866449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fcc975f-291c-4f45-ba48-f9e54c161fdf-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.866657 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.866516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fcc975f-291c-4f45-ba48-f9e54c161fdf-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.866657 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.866584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fcc975f-291c-4f45-ba48-f9e54c161fdf-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.866657 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.866612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvp6c\" (UniqueName: \"kubernetes.io/projected/3fcc975f-291c-4f45-ba48-f9e54c161fdf-kube-api-access-rvp6c\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.866878 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.866690    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fcc975f-291c-4f45-ba48-f9e54c161fdf-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.866878 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.866778    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fcc975f-291c-4f45-ba48-f9e54c161fdf-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.866961 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.866873    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fcc975f-291c-4f45-ba48-f9e54c161fdf-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.868780 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.868758    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fcc975f-291c-4f45-ba48-f9e54c161fdf-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.868892 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.868877    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fcc975f-291c-4f45-ba48-f9e54c161fdf-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.874568 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.874542    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvp6c\" (UniqueName: \"kubernetes.io/projected/3fcc975f-291c-4f45-ba48-f9e54c161fdf-kube-api-access-rvp6c\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kq8z9\" (UID: \"3fcc975f-291c-4f45-ba48-f9e54c161fdf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:45.981093 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:45.981002    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:46.253454 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:46.253410    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"]
Sep 17 03:29:46.254519 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:29:46.254491    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fcc975f_291c_4f45_ba48_f9e54c161fdf.slice/crio-04e17fef4c3a85be6574cce7a5d23dc491987c8d9640896bbce9cc46e04eda69 WatchSource:0}: Error finding container 04e17fef4c3a85be6574cce7a5d23dc491987c8d9640896bbce9cc46e04eda69: Status 404 returned error can't find the container with id 04e17fef4c3a85be6574cce7a5d23dc491987c8d9640896bbce9cc46e04eda69
Sep 17 03:29:46.255807 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:46.255789    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 03:29:46.299331 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:46.299293    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9" event={"ID":"3fcc975f-291c-4f45-ba48-f9e54c161fdf","Type":"ContainerStarted","Data":"04e17fef4c3a85be6574cce7a5d23dc491987c8d9640896bbce9cc46e04eda69"}
Sep 17 03:29:47.308121 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:47.308085    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9" event={"ID":"3fcc975f-291c-4f45-ba48-f9e54c161fdf","Type":"ContainerStarted","Data":"6226458bd843ecebb7883fd4577ee7bdec75f9fb4dd763776a5b6bd44b4c620b"}
Sep 17 03:29:47.325470 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:47.325394    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9" podStartSLOduration=2.158926507 podStartE2EDuration="2.325375644s" totalImagesPullingTime="166.449137ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:29:47.324467569 +0000 UTC m=+615.669193648" watchObservedRunningTime="2026-09-17 03:29:47.325375644 +0000 UTC m=+615.670101720"
Sep 17 03:29:47.538835 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:47.538798    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:29:48.404954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:48.404894    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:29:54.336610 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:54.336524    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:29:55.114681 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:55.114649    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:29:55.981744 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:55.981696    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:55.982192 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:55.981800    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:55.994674 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:55.994647    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:29:56.362562 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:29:56.362534    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kq8z9"
Sep 17 03:30:38.675572 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:30:38.675529    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:30:50.163149 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:30:50.163110    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:24.092315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:24.092237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:24.201625 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:24.201588    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:25.225755 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:25.225705    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:25.318069 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:25.318037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:41.295607 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:41.295571    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:41.396332 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:41.396274    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:41.497544 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:41.497512    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:41.642615 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:41.642584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:41.797642 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:41.797596    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:41.916584 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:41.916496    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:42.020997 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:42.020963    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:42.220622 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:42.220532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:42.732496 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:42.732454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:42.831019 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:42.830990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:43.128767 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:43.128738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:43.254154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:43.254121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:44.612920 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:44.612872    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:44.722466 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:44.722412    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:44.818550 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:44.818515    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:44.921493 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:44.921405    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:45.248900 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:45.248817    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:31:46.385402 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.385368    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"]
Sep 17 03:31:46.389146 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.389124    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:46.392416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.392385    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-297324-serving-cert\""
Sep 17 03:31:46.393059 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.393027    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 03:31:46.393182 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.393070    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-trmz9\""
Sep 17 03:31:46.393182 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.393095    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 17 03:31:46.393182 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.393102    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 17 03:31:46.402004 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.401979    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"]
Sep 17 03:31:46.408667 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.408638    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5bgd\" (UniqueName: \"kubernetes.io/projected/f6fcd273-1108-4a23-856a-4e328f7a7834-kube-api-access-w5bgd\") pod \"maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:46.408782 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.408683    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-maas-api-tls\") pod \"maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:46.408782 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.408746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6fcd273-1108-4a23-856a-4e328f7a7834-openshift-service-ca\") pod \"maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:46.408853 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.408782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-metrics-certs\") pod \"maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:46.423192 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.423152    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:46.509228 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.509193    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5bgd\" (UniqueName: \"kubernetes.io/projected/f6fcd273-1108-4a23-856a-4e328f7a7834-kube-api-access-w5bgd\") pod \"maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:46.509422 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.509235    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-maas-api-tls\") pod \"maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:46.509422 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.509262    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6fcd273-1108-4a23-856a-4e328f7a7834-openshift-service-ca\") pod \"maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:46.509422 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.509291    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-metrics-certs\") pod \"maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:46.509422 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:31:46.509352    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-serving-cert: secret "maas-api-e2e-inf-a-297324-serving-cert" not found
Sep 17 03:31:46.509422 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:31:46.509422    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-maas-api-tls podName:f6fcd273-1108-4a23-856a-4e328f7a7834 nodeName:}" failed. No retries permitted until 2026-09-17 03:31:47.00940548 +0000 UTC m=+735.354131536 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-maas-api-tls") pod "maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d" (UID: "f6fcd273-1108-4a23-856a-4e328f7a7834") : secret "maas-api-e2e-inf-a-297324-serving-cert" not found
Sep 17 03:31:46.509868 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.509849    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6fcd273-1108-4a23-856a-4e328f7a7834-openshift-service-ca\") pod \"maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:46.511821 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.511803    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-metrics-certs\") pod \"maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:46.524265 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.524238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:46.525994 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.525970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5bgd\" (UniqueName: \"kubernetes.io/projected/f6fcd273-1108-4a23-856a-4e328f7a7834-kube-api-access-w5bgd\") pod \"maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:46.675677 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.675591    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:46.925587 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:46.925549    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:47.012794 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:47.012712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-maas-api-tls\") pod \"maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:47.015190 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:47.015155    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-maas-api-tls\") pod \"maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:47.045025 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:47.044992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:47.299630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:47.299551    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:47.578725 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:47.578687    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"]
Sep 17 03:31:47.580718 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:31:47.580680    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6fcd273_1108_4a23_856a_4e328f7a7834.slice/crio-7c4c2f9cc727a9895650ba4b3cce3dc37648a79b6ec003f0860180cd59947ecd WatchSource:0}: Error finding container 7c4c2f9cc727a9895650ba4b3cce3dc37648a79b6ec003f0860180cd59947ecd: Status 404 returned error can't find the container with id 7c4c2f9cc727a9895650ba4b3cce3dc37648a79b6ec003f0860180cd59947ecd
Sep 17 03:31:47.593869 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:47.593838    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:47.689861 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:47.689822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:47.789744 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:47.789703    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:47.910177 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:47.910096    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d" event={"ID":"f6fcd273-1108-4a23-856a-4e328f7a7834","Type":"ContainerStarted","Data":"7c4c2f9cc727a9895650ba4b3cce3dc37648a79b6ec003f0860180cd59947ecd"}
Sep 17 03:31:48.144618 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.144569    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:48.245274 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.245176    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:48.282514 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.282474    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"]
Sep 17 03:31:48.286645 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.286615    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"
Sep 17 03:31:48.288945 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.288700    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-214fea\""
Sep 17 03:31:48.288945 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.288720    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-214fea-dockercfg-vll9t\""
Sep 17 03:31:48.301238 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.301211    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"]
Sep 17 03:31:48.325027 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.324983    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhjz4\" (UniqueName: \"kubernetes.io/projected/2c263c28-777c-43be-917e-d4c2a7712a1c-kube-api-access-nhjz4\") pod \"payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2\" (UID: \"2c263c28-777c-43be-917e-d4c2a7712a1c\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"
Sep 17 03:31:48.325215 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.325060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c263c28-777c-43be-917e-d4c2a7712a1c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2\" (UID: \"2c263c28-777c-43be-917e-d4c2a7712a1c\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"
Sep 17 03:31:48.426274 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.426240    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhjz4\" (UniqueName: \"kubernetes.io/projected/2c263c28-777c-43be-917e-d4c2a7712a1c-kube-api-access-nhjz4\") pod \"payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2\" (UID: \"2c263c28-777c-43be-917e-d4c2a7712a1c\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"
Sep 17 03:31:48.426478 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.426306    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c263c28-777c-43be-917e-d4c2a7712a1c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2\" (UID: \"2c263c28-777c-43be-917e-d4c2a7712a1c\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"
Sep 17 03:31:48.427354 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.427303    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c263c28-777c-43be-917e-d4c2a7712a1c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2\" (UID: \"2c263c28-777c-43be-917e-d4c2a7712a1c\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"
Sep 17 03:31:48.434751 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.434720    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhjz4\" (UniqueName: \"kubernetes.io/projected/2c263c28-777c-43be-917e-d4c2a7712a1c-kube-api-access-nhjz4\") pod \"payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2\" (UID: \"2c263c28-777c-43be-917e-d4c2a7712a1c\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"
Sep 17 03:31:48.606080 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.605938    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"
Sep 17 03:31:48.650028 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.649995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:48.766674 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.766625    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:48.904019 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.903986    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"]
Sep 17 03:31:48.911502 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:48.911416    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:49.223751 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.223661    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:49.311647 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.311610    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"]
Sep 17 03:31:49.313580 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.313555    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:49.319841 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.319229    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"
Sep 17 03:31:49.321563 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.321536    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-1afd34\""
Sep 17 03:31:49.326592 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.326565    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"]
Sep 17 03:31:49.334472 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.334411    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmf64\" (UniqueName: \"kubernetes.io/projected/e45dbaad-e524-4b5a-ae25-d449de969fe2-kube-api-access-jmf64\") pod \"payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq\" (UID: \"e45dbaad-e524-4b5a-ae25-d449de969fe2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"
Sep 17 03:31:49.334599 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.334516    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e45dbaad-e524-4b5a-ae25-d449de969fe2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq\" (UID: \"e45dbaad-e524-4b5a-ae25-d449de969fe2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"
Sep 17 03:31:49.348885 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.348855    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"]
Sep 17 03:31:49.353969 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.353947    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"
Sep 17 03:31:49.355794 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.355771    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-1afd34-dockercfg-rr69r\""
Sep 17 03:31:49.363463 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.363424    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"]
Sep 17 03:31:49.441054 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.440989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e45dbaad-e524-4b5a-ae25-d449de969fe2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq\" (UID: \"e45dbaad-e524-4b5a-ae25-d449de969fe2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"
Sep 17 03:31:49.441285 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.441081    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35aa00e6-c101-4854-9a2a-cad7050f8fd5-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9\" (UID: \"35aa00e6-c101-4854-9a2a-cad7050f8fd5\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"
Sep 17 03:31:49.441285 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.441134    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5rnc\" (UniqueName: \"kubernetes.io/projected/35aa00e6-c101-4854-9a2a-cad7050f8fd5-kube-api-access-f5rnc\") pod \"payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9\" (UID: \"35aa00e6-c101-4854-9a2a-cad7050f8fd5\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"
Sep 17 03:31:49.441285 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.441205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmf64\" (UniqueName: \"kubernetes.io/projected/e45dbaad-e524-4b5a-ae25-d449de969fe2-kube-api-access-jmf64\") pod \"payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq\" (UID: \"e45dbaad-e524-4b5a-ae25-d449de969fe2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"
Sep 17 03:31:49.442038 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.442000    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e45dbaad-e524-4b5a-ae25-d449de969fe2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq\" (UID: \"e45dbaad-e524-4b5a-ae25-d449de969fe2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"
Sep 17 03:31:49.449599 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.449536    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmf64\" (UniqueName: \"kubernetes.io/projected/e45dbaad-e524-4b5a-ae25-d449de969fe2-kube-api-access-jmf64\") pod \"payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq\" (UID: \"e45dbaad-e524-4b5a-ae25-d449de969fe2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"
Sep 17 03:31:49.542681 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.542599    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35aa00e6-c101-4854-9a2a-cad7050f8fd5-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9\" (UID: \"35aa00e6-c101-4854-9a2a-cad7050f8fd5\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"
Sep 17 03:31:49.542681 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.542652    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5rnc\" (UniqueName: \"kubernetes.io/projected/35aa00e6-c101-4854-9a2a-cad7050f8fd5-kube-api-access-f5rnc\") pod \"payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9\" (UID: \"35aa00e6-c101-4854-9a2a-cad7050f8fd5\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"
Sep 17 03:31:49.543343 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.543316    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35aa00e6-c101-4854-9a2a-cad7050f8fd5-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9\" (UID: \"35aa00e6-c101-4854-9a2a-cad7050f8fd5\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"
Sep 17 03:31:49.550252 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.550225    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5rnc\" (UniqueName: \"kubernetes.io/projected/35aa00e6-c101-4854-9a2a-cad7050f8fd5-kube-api-access-f5rnc\") pod \"payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9\" (UID: \"35aa00e6-c101-4854-9a2a-cad7050f8fd5\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"
Sep 17 03:31:49.634030 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.633995    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"
Sep 17 03:31:49.667318 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.667284    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"
Sep 17 03:31:49.754264 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:49.754227    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:49.951785 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:31:49.951747    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c263c28_777c_43be_917e_d4c2a7712a1c.slice/crio-8927eaec860d9e656acb5223a952275693cd3a4d38303d02012c0a1388a4428e WatchSource:0}: Error finding container 8927eaec860d9e656acb5223a952275693cd3a4d38303d02012c0a1388a4428e: Status 404 returned error can't find the container with id 8927eaec860d9e656acb5223a952275693cd3a4d38303d02012c0a1388a4428e
Sep 17 03:31:50.215848 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.215779    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:50.231072 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.231046    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"]
Sep 17 03:31:50.232581 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:31:50.232532    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45dbaad_e524_4b5a_ae25_d449de969fe2.slice/crio-a16a160968d4c715ac67276d1d940377402c601339bce34cea717958f8d2325c WatchSource:0}: Error finding container a16a160968d4c715ac67276d1d940377402c601339bce34cea717958f8d2325c: Status 404 returned error can't find the container with id a16a160968d4c715ac67276d1d940377402c601339bce34cea717958f8d2325c
Sep 17 03:31:50.235837 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.235807    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"]
Sep 17 03:31:50.237177 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:31:50.237155    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35aa00e6_c101_4854_9a2a_cad7050f8fd5.slice/crio-8f623ea76cd0367efbeee68837aed7dff9d537120c0f7f8929920d13e3731c42 WatchSource:0}: Error finding container 8f623ea76cd0367efbeee68837aed7dff9d537120c0f7f8929920d13e3731c42: Status 404 returned error can't find the container with id 8f623ea76cd0367efbeee68837aed7dff9d537120c0f7f8929920d13e3731c42
Sep 17 03:31:50.318320 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.318289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:50.468181 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.468094    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:50.569747 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.569703    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:50.792673 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.792532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:50.892202 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.892104    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:50.932899 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.932859    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq" event={"ID":"e45dbaad-e524-4b5a-ae25-d449de969fe2","Type":"ContainerStarted","Data":"a16a160968d4c715ac67276d1d940377402c601339bce34cea717958f8d2325c"}
Sep 17 03:31:50.935879 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.935840    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d" event={"ID":"f6fcd273-1108-4a23-856a-4e328f7a7834","Type":"ContainerStarted","Data":"5aa1723dccf6dbb58e731b0355e855d844c53823af8c38e8a4f86d406d547c12"}
Sep 17 03:31:50.936489 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.936467    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:50.938396 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.938367    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2" event={"ID":"2c263c28-777c-43be-917e-d4c2a7712a1c","Type":"ContainerStarted","Data":"8927eaec860d9e656acb5223a952275693cd3a4d38303d02012c0a1388a4428e"}
Sep 17 03:31:50.940110 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.940026    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9" event={"ID":"35aa00e6-c101-4854-9a2a-cad7050f8fd5","Type":"ContainerStarted","Data":"8f623ea76cd0367efbeee68837aed7dff9d537120c0f7f8929920d13e3731c42"}
Sep 17 03:31:50.957074 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:50.954981    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d" podStartSLOduration=2.5116987330000002 podStartE2EDuration="4.954959907s" totalImagesPullingTime="2.443261174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:31:50.953836436 +0000 UTC m=+739.298562518" watchObservedRunningTime="2026-09-17 03:31:50.954959907 +0000 UTC m=+739.299685987"
Sep 17 03:31:51.040785 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:51.040743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:51.187653 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:51.187617    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:51.329041 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:51.329006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:51.626420 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:51.626386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:51.947138 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:51.947044    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2" event={"ID":"2c263c28-777c-43be-917e-d4c2a7712a1c","Type":"ContainerStarted","Data":"e5d55ee6da3c6177e3b98383e736ce15711a07219253640eea69b2feaece4c91"}
Sep 17 03:31:51.947138 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:51.947119    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"
Sep 17 03:31:51.948814 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:51.948786    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9" event={"ID":"35aa00e6-c101-4854-9a2a-cad7050f8fd5","Type":"ContainerStarted","Data":"cf3529f161676277bbc48c8569aace1ceeaa8f2237c0e137cf791f665df6d95a"}
Sep 17 03:31:51.948943 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:51.948829    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"
Sep 17 03:31:51.950474 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:51.950454    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq" event={"ID":"e45dbaad-e524-4b5a-ae25-d449de969fe2","Type":"ContainerStarted","Data":"3a2aaace0cb450bf99a9561d25638495944eaf1e38f8dd2a4fba17fd30c66363"}
Sep 17 03:31:51.950597 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:51.950585    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"
Sep 17 03:31:51.964076 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:51.964033    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2" podStartSLOduration=2.697949229 podStartE2EDuration="3.964017007s" totalImagesPullingTime="1.266067778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:31:51.962990486 +0000 UTC m=+740.307716565" watchObservedRunningTime="2026-09-17 03:31:51.964017007 +0000 UTC m=+740.308743085"
Sep 17 03:31:51.980587 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:51.980526    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9" podStartSLOduration=1.933724161 podStartE2EDuration="2.980509227s" totalImagesPullingTime="1.046785066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:31:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:31:51.977987264 +0000 UTC m=+740.322713367" watchObservedRunningTime="2026-09-17 03:31:51.980509227 +0000 UTC m=+740.325235306"
Sep 17 03:31:51.993660 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:51.993600    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq" podStartSLOduration=1.963131861 podStartE2EDuration="2.993578403s" totalImagesPullingTime="1.030446542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:31:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:31:51.993205228 +0000 UTC m=+740.337931318" watchObservedRunningTime="2026-09-17 03:31:51.993578403 +0000 UTC m=+740.338304480"
Sep 17 03:31:54.192344 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:54.192306    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:54.868079 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:54.868032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:54.954894 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:54.954862    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:55.736771 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:55.736741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:56.347147 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:56.347113    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:56.445628 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:56.445593    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:57.033120 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:57.033084    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:57.960780 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:57.960751    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:31:58.561791 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:58.561758    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:59.379156 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:59.379116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:59.858088 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:59.858042    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:31:59.947350 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:31:59.947320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:00.049543 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:00.049510    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:00.256132 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:00.256044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:00.349377 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:00.349343    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:00.445287 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:00.445253    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:00.649511 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:00.649477    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:00.751029 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:00.750996    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:00.910634 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:00.910539    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:01.013514 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.013474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:01.039740 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.039707    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"]
Sep 17 03:32:01.043110 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.043094    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"
Sep 17 03:32:01.047146 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.047125    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-f6a774\""
Sep 17 03:32:01.056882 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.056857    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"]
Sep 17 03:32:01.146631 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.146590    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwnht\" (UniqueName: \"kubernetes.io/projected/ad3a4de5-016e-4816-85e4-feb75d050596-kube-api-access-rwnht\") pod \"payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9\" (UID: \"ad3a4de5-016e-4816-85e4-feb75d050596\") " pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"
Sep 17 03:32:01.146837 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.146640    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad3a4de5-016e-4816-85e4-feb75d050596-plugins-config-volume\") pod \"payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9\" (UID: \"ad3a4de5-016e-4816-85e4-feb75d050596\") " pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"
Sep 17 03:32:01.148930 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.148905    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:32:01.248052 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.247952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwnht\" (UniqueName: \"kubernetes.io/projected/ad3a4de5-016e-4816-85e4-feb75d050596-kube-api-access-rwnht\") pod \"payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9\" (UID: \"ad3a4de5-016e-4816-85e4-feb75d050596\") " pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"
Sep 17 03:32:01.248052 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.247991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad3a4de5-016e-4816-85e4-feb75d050596-plugins-config-volume\") pod \"payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9\" (UID: \"ad3a4de5-016e-4816-85e4-feb75d050596\") " pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"
Sep 17 03:32:01.248753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.248725    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad3a4de5-016e-4816-85e4-feb75d050596-plugins-config-volume\") pod \"payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9\" (UID: \"ad3a4de5-016e-4816-85e4-feb75d050596\") " pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"
Sep 17 03:32:01.255788 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.255760    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwnht\" (UniqueName: \"kubernetes.io/projected/ad3a4de5-016e-4816-85e4-feb75d050596-kube-api-access-rwnht\") pod \"payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9\" (UID: \"ad3a4de5-016e-4816-85e4-feb75d050596\") " pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"
Sep 17 03:32:01.353820 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.353762    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"
Sep 17 03:32:01.440403 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.440369    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:01.561733 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.561697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:01.658181 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:01.658154    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"]
Sep 17 03:32:01.661000 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:32:01.660964    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad3a4de5_016e_4816_85e4_feb75d050596.slice/crio-be7ea1a65e4186a953d42d4ef730d3407e6d92ae620d5c87b2c4acc592d7ec51 WatchSource:0}: Error finding container be7ea1a65e4186a953d42d4ef730d3407e6d92ae620d5c87b2c4acc592d7ec51: Status 404 returned error can't find the container with id be7ea1a65e4186a953d42d4ef730d3407e6d92ae620d5c87b2c4acc592d7ec51
Sep 17 03:32:02.007412 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:02.007309    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9" event={"ID":"ad3a4de5-016e-4816-85e4-feb75d050596","Type":"ContainerStarted","Data":"00a9bdded79097f0f8f23de787d009c155836d4db2acc74b60341425ddfd120f"}
Sep 17 03:32:02.007412 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:02.007373    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9" event={"ID":"ad3a4de5-016e-4816-85e4-feb75d050596","Type":"ContainerStarted","Data":"be7ea1a65e4186a953d42d4ef730d3407e6d92ae620d5c87b2c4acc592d7ec51"}
Sep 17 03:32:02.007946 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:02.007503    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"
Sep 17 03:32:02.022065 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:02.022008    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9" podStartSLOduration=1.021992597 podStartE2EDuration="1.021992597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:32:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:32:02.020786656 +0000 UTC m=+750.365512751" watchObservedRunningTime="2026-09-17 03:32:02.021992597 +0000 UTC m=+750.366718675"
Sep 17 03:32:02.127183 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:02.127147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:02.227850 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:02.227819    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:02.360237 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:02.360199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:02.928333 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:02.928300    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:02.957417 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:02.957386    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"
Sep 17 03:32:02.957591 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:02.957488    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"
Sep 17 03:32:02.957689 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:02.957666    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"
Sep 17 03:32:03.218811 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:03.218714    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:03.731671 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:03.731638    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:04.530144 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:04.530109    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:04.629749 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:04.629712    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:04.837444 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:04.837405    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:04.982924 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:04.982891    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:05.081157 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:05.081118    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:05.180682 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:05.180587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:05.485684 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:05.485595    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:06.449041 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:06.449006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:06.557695 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:06.557664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:06.653990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:06.653955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:06.855286 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:06.855253    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:07.501966 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:07.501934    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:08.468890 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:08.468856    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:09.628693 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:09.628658    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:09.916500 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:09.916406    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:10.016401 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:10.016362    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:10.391331 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:10.391291    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:11.598050 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:11.598017    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:11.784604 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:11.784571    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:13.019413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:13.019384    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"
Sep 17 03:32:13.153185 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:13.153149    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:13.220771 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:13.220711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:13.308560 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:13.308527    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:13.414195 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:13.414163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:13.550080 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:13.550040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:13.618347 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:13.618259    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:13.760225 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:13.760195    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:13.852179 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:13.852116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:14.832346 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:14.832313    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:15.693215 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:15.693174    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:16.526733 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:16.526699    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:16.638915 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:16.638879    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:32:17.447377 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:17.447341    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:18.437165 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:18.437132    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:19.210565 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:19.210537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:19.668198 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:19.668164    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:19.897841 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:19.897799    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"]
Sep 17 03:32:19.903223 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:19.903199    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:19.905347 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:19.905321    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3d0c3c-serving-cert\""
Sep 17 03:32:19.912895 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:19.912869    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"]
Sep 17 03:32:19.938673 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:19.938545    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:20.019031 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.018999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/674d2736-fcb8-420e-b66a-dfd58993dcbf-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:20.019198 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.019069    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/674d2736-fcb8-420e-b66a-dfd58993dcbf-metrics-certs\") pod \"maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:20.019198 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.019094    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c45rq\" (UniqueName: \"kubernetes.io/projected/674d2736-fcb8-420e-b66a-dfd58993dcbf-kube-api-access-c45rq\") pod \"maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:20.019198 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.019113    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/674d2736-fcb8-420e-b66a-dfd58993dcbf-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:20.043515 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.043475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:20.119620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.119590    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/674d2736-fcb8-420e-b66a-dfd58993dcbf-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:20.119809 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.119667    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/674d2736-fcb8-420e-b66a-dfd58993dcbf-metrics-certs\") pod \"maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:20.119809 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.119696    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45rq\" (UniqueName: \"kubernetes.io/projected/674d2736-fcb8-420e-b66a-dfd58993dcbf-kube-api-access-c45rq\") pod \"maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:20.119809 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.119714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/674d2736-fcb8-420e-b66a-dfd58993dcbf-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:20.120422 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.120400    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/674d2736-fcb8-420e-b66a-dfd58993dcbf-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:20.122082 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.122060    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/674d2736-fcb8-420e-b66a-dfd58993dcbf-metrics-certs\") pod \"maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:20.122170 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.122101    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/674d2736-fcb8-420e-b66a-dfd58993dcbf-maas-api-tls\") pod \"maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:20.127524 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.127501    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c45rq\" (UniqueName: \"kubernetes.io/projected/674d2736-fcb8-420e-b66a-dfd58993dcbf-kube-api-access-c45rq\") pod \"maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:20.139969 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.139926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:20.214892 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.214799    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:20.237170 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.237137    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:20.442200 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.442166    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:20.490454 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.490405    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"]
Sep 17 03:32:20.492991 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:32:20.492959    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod674d2736_fcb8_420e_b66a_dfd58993dcbf.slice/crio-042383a1c38198df4b5c197b599e654b939bde2ab4cd9a5d97ffab031276ba1e WatchSource:0}: Error finding container 042383a1c38198df4b5c197b599e654b939bde2ab4cd9a5d97ffab031276ba1e: Status 404 returned error can't find the container with id 042383a1c38198df4b5c197b599e654b939bde2ab4cd9a5d97ffab031276ba1e
Sep 17 03:32:20.542149 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.542108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:20.549937 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.549905    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"]
Sep 17 03:32:20.552873 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.552853    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"
Sep 17 03:32:20.555067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.555049    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-3d0c3c\""
Sep 17 03:32:20.562853 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.562826    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"]
Sep 17 03:32:20.711031 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.711000    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"]
Sep 17 03:32:20.713685 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.713662    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"
Sep 17 03:32:20.716204 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.716175    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-3d0c3c-dockercfg-gh628\""
Sep 17 03:32:20.723796 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.723764    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8a47a4e-01e9-4fde-973d-8106d545b716-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62\" (UID: \"f8a47a4e-01e9-4fde-973d-8106d545b716\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"
Sep 17 03:32:20.723940 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.723858    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv6cq\" (UniqueName: \"kubernetes.io/projected/f8a47a4e-01e9-4fde-973d-8106d545b716-kube-api-access-hv6cq\") pod \"payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62\" (UID: \"f8a47a4e-01e9-4fde-973d-8106d545b716\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"
Sep 17 03:32:20.725802 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.725777    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"]
Sep 17 03:32:20.824545 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.824519    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkv2j\" (UniqueName: \"kubernetes.io/projected/813bbf20-e4e8-4476-a390-e374a0e3c2db-kube-api-access-xkv2j\") pod \"payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr\" (UID: \"813bbf20-e4e8-4476-a390-e374a0e3c2db\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"
Sep 17 03:32:20.824682 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.824564    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv6cq\" (UniqueName: \"kubernetes.io/projected/f8a47a4e-01e9-4fde-973d-8106d545b716-kube-api-access-hv6cq\") pod \"payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62\" (UID: \"f8a47a4e-01e9-4fde-973d-8106d545b716\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"
Sep 17 03:32:20.824730 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.824693    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8a47a4e-01e9-4fde-973d-8106d545b716-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62\" (UID: \"f8a47a4e-01e9-4fde-973d-8106d545b716\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"
Sep 17 03:32:20.824771 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.824746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/813bbf20-e4e8-4476-a390-e374a0e3c2db-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr\" (UID: \"813bbf20-e4e8-4476-a390-e374a0e3c2db\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"
Sep 17 03:32:20.825470 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.825451    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8a47a4e-01e9-4fde-973d-8106d545b716-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62\" (UID: \"f8a47a4e-01e9-4fde-973d-8106d545b716\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"
Sep 17 03:32:20.832283 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.832260    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv6cq\" (UniqueName: \"kubernetes.io/projected/f8a47a4e-01e9-4fde-973d-8106d545b716-kube-api-access-hv6cq\") pod \"payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62\" (UID: \"f8a47a4e-01e9-4fde-973d-8106d545b716\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"
Sep 17 03:32:20.865273 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.865242    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"
Sep 17 03:32:20.926160 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.926128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/813bbf20-e4e8-4476-a390-e374a0e3c2db-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr\" (UID: \"813bbf20-e4e8-4476-a390-e374a0e3c2db\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"
Sep 17 03:32:20.926325 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.926188    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkv2j\" (UniqueName: \"kubernetes.io/projected/813bbf20-e4e8-4476-a390-e374a0e3c2db-kube-api-access-xkv2j\") pod \"payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr\" (UID: \"813bbf20-e4e8-4476-a390-e374a0e3c2db\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"
Sep 17 03:32:20.926831 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.926808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/813bbf20-e4e8-4476-a390-e374a0e3c2db-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr\" (UID: \"813bbf20-e4e8-4476-a390-e374a0e3c2db\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"
Sep 17 03:32:20.935006 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:20.934984    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkv2j\" (UniqueName: \"kubernetes.io/projected/813bbf20-e4e8-4476-a390-e374a0e3c2db-kube-api-access-xkv2j\") pod \"payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr\" (UID: \"813bbf20-e4e8-4476-a390-e374a0e3c2db\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"
Sep 17 03:32:21.024956 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:21.024920    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"
Sep 17 03:32:21.089046 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:21.089003    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:21.119886 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:21.119850    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg" event={"ID":"674d2736-fcb8-420e-b66a-dfd58993dcbf","Type":"ContainerStarted","Data":"9b9295f16dc002ccf18351349a21794051924aefbdb21b2012279273e827bdea"}
Sep 17 03:32:21.120130 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:21.119909    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg" event={"ID":"674d2736-fcb8-420e-b66a-dfd58993dcbf","Type":"ContainerStarted","Data":"042383a1c38198df4b5c197b599e654b939bde2ab4cd9a5d97ffab031276ba1e"}
Sep 17 03:32:21.120130 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:21.120058    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:21.153729 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:21.153666    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg" podStartSLOduration=1.8569800619999999 podStartE2EDuration="2.153647697s" totalImagesPullingTime="296.667635ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:32:21.151703025 +0000 UTC m=+769.496429104" watchObservedRunningTime="2026-09-17 03:32:21.153647697 +0000 UTC m=+769.498373776"
Sep 17 03:32:21.159111 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:21.159080    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"]
Sep 17 03:32:21.159294 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:32:21.159225    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a47a4e_01e9_4fde_973d_8106d545b716.slice/crio-59ec9780d6d27bdee18c73171090d9ba5eaafee1266cb2ff1fe095d1d0b71672 WatchSource:0}: Error finding container 59ec9780d6d27bdee18c73171090d9ba5eaafee1266cb2ff1fe095d1d0b71672: Status 404 returned error can't find the container with id 59ec9780d6d27bdee18c73171090d9ba5eaafee1266cb2ff1fe095d1d0b71672
Sep 17 03:32:21.305793 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:21.305760    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"]
Sep 17 03:32:21.306794 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:32:21.306764    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod813bbf20_e4e8_4476_a390_e374a0e3c2db.slice/crio-59b8bdc48b6211111549a8cbed210858dcecda59a600a78c29efa807c64e0719 WatchSource:0}: Error finding container 59b8bdc48b6211111549a8cbed210858dcecda59a600a78c29efa807c64e0719: Status 404 returned error can't find the container with id 59b8bdc48b6211111549a8cbed210858dcecda59a600a78c29efa807c64e0719
Sep 17 03:32:22.127412 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:22.127372    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr" event={"ID":"813bbf20-e4e8-4476-a390-e374a0e3c2db","Type":"ContainerStarted","Data":"0d2540a5019665f5e91ba8dcf98f959d88eed853e3faa7888ba86fcaed046d78"}
Sep 17 03:32:22.127867 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:22.127454    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr" event={"ID":"813bbf20-e4e8-4476-a390-e374a0e3c2db","Type":"ContainerStarted","Data":"59b8bdc48b6211111549a8cbed210858dcecda59a600a78c29efa807c64e0719"}
Sep 17 03:32:22.127867 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:22.127572    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"
Sep 17 03:32:22.129084 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:22.129058    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62" event={"ID":"f8a47a4e-01e9-4fde-973d-8106d545b716","Type":"ContainerStarted","Data":"827b7f67f8f39c0c250739ef609a14a9ad1e60cfba9e3a55b8d501a8037940c6"}
Sep 17 03:32:22.129227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:22.129105    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62" event={"ID":"f8a47a4e-01e9-4fde-973d-8106d545b716","Type":"ContainerStarted","Data":"59ec9780d6d27bdee18c73171090d9ba5eaafee1266cb2ff1fe095d1d0b71672"}
Sep 17 03:32:22.129227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:22.129176    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"
Sep 17 03:32:22.147230 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:22.147180    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr" podStartSLOduration=2.147165046 podStartE2EDuration="2.147165046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:32:22.14530202 +0000 UTC m=+770.490028099" watchObservedRunningTime="2026-09-17 03:32:22.147165046 +0000 UTC m=+770.491891187"
Sep 17 03:32:22.161347 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:22.161288    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62" podStartSLOduration=2.161273887 podStartE2EDuration="2.161273887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:32:22.159857185 +0000 UTC m=+770.504583265" watchObservedRunningTime="2026-09-17 03:32:22.161273887 +0000 UTC m=+770.505999965"
Sep 17 03:32:22.169140 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:22.169104    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:23.112953 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:23.112915    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:23.961057 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:23.961021    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:24.863466 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:24.863422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:25.659354 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:25.659312    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:26.564240 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:26.564207    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:27.135303 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:27.135274    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:32:27.263142 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:27.263110    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:32.824832 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:32.824796    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:32:33.141891 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:33.141806    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"
Sep 17 03:32:33.141891 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:33.141875    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"
Sep 17 03:32:37.827364 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:37.827322    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:38.502951 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:38.502918    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:38.600892 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:38.600837    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:38.669255 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:38.669215    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:38.737945 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:38.737893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:38.825120 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:38.825086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:39.028771 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:39.028736    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:39.271214 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:39.271130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:39.373021 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:39.372990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:39.491778 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:39.491742    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:39.813396 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:39.813358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:40.031207 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:40.031177    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:40.436235 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:40.436203    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:40.816680 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:40.816643    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:41.682276 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:41.682243    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:42.098694 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:42.098664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:42.595374 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:42.595332    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:42.697779 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:42.697743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:42.796526 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:42.796494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:43.530294 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:43.530255    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:44.282944 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:44.282914    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:44.384015 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:44.383979    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:44.831646 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:44.831614    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:44.956568 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:44.956536    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:45.137192 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.137108    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"]
Sep 17 03:32:45.140011 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.139988    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"
Sep 17 03:32:45.142170 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.142130    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-0ae566a4-dockercfg-qx5r4\""
Sep 17 03:32:45.142318 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.142187    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-0ae566a4\""
Sep 17 03:32:45.149416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.149380    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"]
Sep 17 03:32:45.247083 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.247042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpf2k\" (UniqueName: \"kubernetes.io/projected/bec5fa84-e95d-415c-a698-b41703c5fce5-kube-api-access-wpf2k\") pod \"payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5\" (UID: \"bec5fa84-e95d-415c-a698-b41703c5fce5\") " pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"
Sep 17 03:32:45.247083 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.247087    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bec5fa84-e95d-415c-a698-b41703c5fce5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5\" (UID: \"bec5fa84-e95d-415c-a698-b41703c5fce5\") " pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"
Sep 17 03:32:45.333195 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.333163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:45.348208 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.348180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpf2k\" (UniqueName: \"kubernetes.io/projected/bec5fa84-e95d-415c-a698-b41703c5fce5-kube-api-access-wpf2k\") pod \"payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5\" (UID: \"bec5fa84-e95d-415c-a698-b41703c5fce5\") " pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"
Sep 17 03:32:45.348368 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.348217    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bec5fa84-e95d-415c-a698-b41703c5fce5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5\" (UID: \"bec5fa84-e95d-415c-a698-b41703c5fce5\") " pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"
Sep 17 03:32:45.348939 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.348919    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bec5fa84-e95d-415c-a698-b41703c5fce5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5\" (UID: \"bec5fa84-e95d-415c-a698-b41703c5fce5\") " pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"
Sep 17 03:32:45.358684 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.358655    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpf2k\" (UniqueName: \"kubernetes.io/projected/bec5fa84-e95d-415c-a698-b41703c5fce5-kube-api-access-wpf2k\") pod \"payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5\" (UID: \"bec5fa84-e95d-415c-a698-b41703c5fce5\") " pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"
Sep 17 03:32:45.455347 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.455252    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"
Sep 17 03:32:45.731330 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.731304    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"]
Sep 17 03:32:45.733575 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:32:45.733543    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbec5fa84_e95d_415c_a698_b41703c5fce5.slice/crio-d5d9dddba16df504d6bcf657a382bb4d65f9c59962a753d8d080cd01e666be47 WatchSource:0}: Error finding container d5d9dddba16df504d6bcf657a382bb4d65f9c59962a753d8d080cd01e666be47: Status 404 returned error can't find the container with id d5d9dddba16df504d6bcf657a382bb4d65f9c59962a753d8d080cd01e666be47
Sep 17 03:32:45.955668 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:45.955631    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:32:46.275757 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:46.275713    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5" event={"ID":"bec5fa84-e95d-415c-a698-b41703c5fce5","Type":"ContainerStarted","Data":"389bc670427bdfb0df83ee95f9be888077d8b0ab737015e9edbc9eb8e6f338e6"}
Sep 17 03:32:46.275977 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:46.275781    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5" event={"ID":"bec5fa84-e95d-415c-a698-b41703c5fce5","Type":"ContainerStarted","Data":"d5d9dddba16df504d6bcf657a382bb4d65f9c59962a753d8d080cd01e666be47"}
Sep 17 03:32:46.276105 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:46.276081    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"
Sep 17 03:32:46.292038 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:46.291974    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5" podStartSLOduration=1.29195627 podStartE2EDuration="1.29195627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:32:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:32:46.290779958 +0000 UTC m=+794.635506037" watchObservedRunningTime="2026-09-17 03:32:46.29195627 +0000 UTC m=+794.636682424"
Sep 17 03:32:46.357458 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:46.357398    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:47.245156 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:47.245121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:48.129582 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:48.129540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:48.841075 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:48.841042    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:48.938671 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:48.938632    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:49.753442 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:49.753302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:50.564720 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:50.564689    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:57.283698 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:57.283612    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"
Sep 17 03:32:58.133091 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:58.133051    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:32:58.201839 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:32:58.201804    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:05.092531 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:05.092495    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:33:06.688623 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:06.688589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:07.386131 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:07.386099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:08.111462 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:08.111421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:08.890753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:08.890723    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:09.619148 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:09.619111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:10.295688 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:10.295655    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:11.119251 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:11.119201    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:11.837110 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:11.837076    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:12.389723 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:12.389690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:12.751770 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:12.751667    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:13.725348 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:13.725310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:13.894395 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:13.894356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:14.334174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.334138    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:14.359026 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.358995    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"]
Sep 17 03:33:14.359773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.359592    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5" podUID="bec5fa84-e95d-415c-a698-b41703c5fce5" containerName="payload-processing" containerID="cri-o://389bc670427bdfb0df83ee95f9be888077d8b0ab737015e9edbc9eb8e6f338e6" gracePeriod=30
Sep 17 03:33:14.436014 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.435986    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:14.536935 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.536905    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:14.605641 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.605586    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:14.617518 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.617494    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"
Sep 17 03:33:14.688083 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.688044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:14.704797 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.704769    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bec5fa84-e95d-415c-a698-b41703c5fce5-plugins-config-volume\") pod \"bec5fa84-e95d-415c-a698-b41703c5fce5\" (UID: \"bec5fa84-e95d-415c-a698-b41703c5fce5\") "
Sep 17 03:33:14.704934 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.704858    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpf2k\" (UniqueName: \"kubernetes.io/projected/bec5fa84-e95d-415c-a698-b41703c5fce5-kube-api-access-wpf2k\") pod \"bec5fa84-e95d-415c-a698-b41703c5fce5\" (UID: \"bec5fa84-e95d-415c-a698-b41703c5fce5\") "
Sep 17 03:33:14.705203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.705179    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bec5fa84-e95d-415c-a698-b41703c5fce5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bec5fa84-e95d-415c-a698-b41703c5fce5" (UID: "bec5fa84-e95d-415c-a698-b41703c5fce5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:33:14.707200 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.707175    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bec5fa84-e95d-415c-a698-b41703c5fce5-kube-api-access-wpf2k" (OuterVolumeSpecName: "kube-api-access-wpf2k") pod "bec5fa84-e95d-415c-a698-b41703c5fce5" (UID: "bec5fa84-e95d-415c-a698-b41703c5fce5"). InnerVolumeSpecName "kube-api-access-wpf2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:33:14.806041 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.806015    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bec5fa84-e95d-415c-a698-b41703c5fce5-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:33:14.806041 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:14.806038    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpf2k\" (UniqueName: \"kubernetes.io/projected/bec5fa84-e95d-415c-a698-b41703c5fce5-kube-api-access-wpf2k\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:33:15.018223 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:15.018113    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:15.127732 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:15.127699    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:15.441444 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:15.441400    2576 generic.go:356] "Generic (PLEG): container finished" podID="bec5fa84-e95d-415c-a698-b41703c5fce5" containerID="389bc670427bdfb0df83ee95f9be888077d8b0ab737015e9edbc9eb8e6f338e6" exitCode=0
Sep 17 03:33:15.441618 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:15.441452    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5" event={"ID":"bec5fa84-e95d-415c-a698-b41703c5fce5","Type":"ContainerDied","Data":"389bc670427bdfb0df83ee95f9be888077d8b0ab737015e9edbc9eb8e6f338e6"}
Sep 17 03:33:15.441618 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:15.441470    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"
Sep 17 03:33:15.441618 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:15.441494    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5" event={"ID":"bec5fa84-e95d-415c-a698-b41703c5fce5","Type":"ContainerDied","Data":"d5d9dddba16df504d6bcf657a382bb4d65f9c59962a753d8d080cd01e666be47"}
Sep 17 03:33:15.441618 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:15.441515    2576 scope.go:117] "RemoveContainer" containerID="389bc670427bdfb0df83ee95f9be888077d8b0ab737015e9edbc9eb8e6f338e6"
Sep 17 03:33:15.451256 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:15.451235    2576 scope.go:117] "RemoveContainer" containerID="389bc670427bdfb0df83ee95f9be888077d8b0ab737015e9edbc9eb8e6f338e6"
Sep 17 03:33:15.451607 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:33:15.451581    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"389bc670427bdfb0df83ee95f9be888077d8b0ab737015e9edbc9eb8e6f338e6\": container with ID starting with 389bc670427bdfb0df83ee95f9be888077d8b0ab737015e9edbc9eb8e6f338e6 not found: ID does not exist" containerID="389bc670427bdfb0df83ee95f9be888077d8b0ab737015e9edbc9eb8e6f338e6"
Sep 17 03:33:15.451682 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:15.451620    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"389bc670427bdfb0df83ee95f9be888077d8b0ab737015e9edbc9eb8e6f338e6"} err="failed to get container status \"389bc670427bdfb0df83ee95f9be888077d8b0ab737015e9edbc9eb8e6f338e6\": rpc error: code = NotFound desc = could not find container \"389bc670427bdfb0df83ee95f9be888077d8b0ab737015e9edbc9eb8e6f338e6\": container with ID starting with 389bc670427bdfb0df83ee95f9be888077d8b0ab737015e9edbc9eb8e6f338e6 not found: ID does not exist"
Sep 17 03:33:15.494774 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:15.494735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:15.494944 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:15.494819    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"]
Sep 17 03:33:15.497034 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:15.497012    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0ae566a4-6b55ccdbc8-qtjt5"]
Sep 17 03:33:15.604660 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:15.604608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:16.140651 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:16.140615    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:16.295241 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:16.295198    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bec5fa84-e95d-415c-a698-b41703c5fce5" path="/var/lib/kubelet/pods/bec5fa84-e95d-415c-a698-b41703c5fce5/volumes"
Sep 17 03:33:17.109812 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:17.109780    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:17.733973 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:17.733941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:17.820982 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:17.820953    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:17.922785 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:17.922747    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:18.833024 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:18.832995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:19.525046 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:19.524994    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:19.888563 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:19.888525    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:33:20.424777 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:20.424745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:20.525440 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:20.525393    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:20.625362 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:20.625290    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:21.059274 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.059243    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:21.220713 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.220680    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:21.394296 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.394185    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"]
Sep 17 03:33:21.394860 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.394838    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bec5fa84-e95d-415c-a698-b41703c5fce5" containerName="payload-processing"
Sep 17 03:33:21.394953 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.394864    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bec5fa84-e95d-415c-a698-b41703c5fce5" containerName="payload-processing"
Sep 17 03:33:21.394953 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.394947    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bec5fa84-e95d-415c-a698-b41703c5fce5" containerName="payload-processing"
Sep 17 03:33:21.397720 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.397701    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"
Sep 17 03:33:21.399845 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.399820    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-3b91ce2e-dockercfg-fgp9m\""
Sep 17 03:33:21.399966 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.399834    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-3b91ce2e\""
Sep 17 03:33:21.407640 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.407618    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"]
Sep 17 03:33:21.568706 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.568666    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41a5703d-e221-4a31-ab30-f734e9f5b04a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d\" (UID: \"41a5703d-e221-4a31-ab30-f734e9f5b04a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"
Sep 17 03:33:21.568881 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.568718    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn2qp\" (UniqueName: \"kubernetes.io/projected/41a5703d-e221-4a31-ab30-f734e9f5b04a-kube-api-access-hn2qp\") pod \"payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d\" (UID: \"41a5703d-e221-4a31-ab30-f734e9f5b04a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"
Sep 17 03:33:21.635100 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.635066    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:21.669504 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.669386    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41a5703d-e221-4a31-ab30-f734e9f5b04a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d\" (UID: \"41a5703d-e221-4a31-ab30-f734e9f5b04a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"
Sep 17 03:33:21.669504 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.669477    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn2qp\" (UniqueName: \"kubernetes.io/projected/41a5703d-e221-4a31-ab30-f734e9f5b04a-kube-api-access-hn2qp\") pod \"payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d\" (UID: \"41a5703d-e221-4a31-ab30-f734e9f5b04a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"
Sep 17 03:33:21.670102 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.670081    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41a5703d-e221-4a31-ab30-f734e9f5b04a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d\" (UID: \"41a5703d-e221-4a31-ab30-f734e9f5b04a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"
Sep 17 03:33:21.678049 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.678016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn2qp\" (UniqueName: \"kubernetes.io/projected/41a5703d-e221-4a31-ab30-f734e9f5b04a-kube-api-access-hn2qp\") pod \"payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d\" (UID: \"41a5703d-e221-4a31-ab30-f734e9f5b04a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"
Sep 17 03:33:21.711580 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.711541    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"
Sep 17 03:33:21.988604 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:21.988576    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"]
Sep 17 03:33:21.988604 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:33:21.988586    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a5703d_e221_4a31_ab30_f734e9f5b04a.slice/crio-2d8c8e9722e7533363899351dc50d46b6b1d7abd071b4ddaa6adac663de19232 WatchSource:0}: Error finding container 2d8c8e9722e7533363899351dc50d46b6b1d7abd071b4ddaa6adac663de19232: Status 404 returned error can't find the container with id 2d8c8e9722e7533363899351dc50d46b6b1d7abd071b4ddaa6adac663de19232
Sep 17 03:33:22.325821 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:22.325784    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:22.486654 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:22.486615    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d" event={"ID":"41a5703d-e221-4a31-ab30-f734e9f5b04a","Type":"ContainerStarted","Data":"7c9e60c56870ee4dbb663aa0dc8c8040acd186de25a8a6b35293bdf08858461c"}
Sep 17 03:33:22.486821 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:22.486680    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d" event={"ID":"41a5703d-e221-4a31-ab30-f734e9f5b04a","Type":"ContainerStarted","Data":"2d8c8e9722e7533363899351dc50d46b6b1d7abd071b4ddaa6adac663de19232"}
Sep 17 03:33:22.486821 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:22.486710    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"
Sep 17 03:33:22.503835 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:22.503776    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d" podStartSLOduration=1.503760449 podStartE2EDuration="1.503760449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:33:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:33:22.502589284 +0000 UTC m=+830.847315364" watchObservedRunningTime="2026-09-17 03:33:22.503760449 +0000 UTC m=+830.848486562"
Sep 17 03:33:23.125931 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:23.125901    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:27.792220 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:27.792178    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:33.495412 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:33.495381    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"
Sep 17 03:33:42.814111 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:42.814075    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:43.583764 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:43.583734    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:44.231382 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:44.231344    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:33:44.402256 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:44.402217    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:45.346451 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:45.346386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:46.132280 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:46.132249    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:46.937656 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:46.937624    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:47.817659 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:47.817627    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:48.700496 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:48.700461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:49.537788 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:49.537743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:50.288207 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.288171    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:50.322213 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.322174    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"]
Sep 17 03:33:50.322405 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.322382    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d" podUID="41a5703d-e221-4a31-ab30-f734e9f5b04a" containerName="payload-pre-processing" containerID="cri-o://7c9e60c56870ee4dbb663aa0dc8c8040acd186de25a8a6b35293bdf08858461c" gracePeriod=30
Sep 17 03:33:50.388599 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.388558    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:50.491762 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.491711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:50.566383 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.566352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:50.577032 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.577008    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"
Sep 17 03:33:50.652306 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.652274    2576 generic.go:356] "Generic (PLEG): container finished" podID="41a5703d-e221-4a31-ab30-f734e9f5b04a" containerID="7c9e60c56870ee4dbb663aa0dc8c8040acd186de25a8a6b35293bdf08858461c" exitCode=0
Sep 17 03:33:50.652516 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.652323    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"
Sep 17 03:33:50.652516 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.652356    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d" event={"ID":"41a5703d-e221-4a31-ab30-f734e9f5b04a","Type":"ContainerDied","Data":"7c9e60c56870ee4dbb663aa0dc8c8040acd186de25a8a6b35293bdf08858461c"}
Sep 17 03:33:50.652516 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.652414    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d" event={"ID":"41a5703d-e221-4a31-ab30-f734e9f5b04a","Type":"ContainerDied","Data":"2d8c8e9722e7533363899351dc50d46b6b1d7abd071b4ddaa6adac663de19232"}
Sep 17 03:33:50.652516 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.652467    2576 scope.go:117] "RemoveContainer" containerID="7c9e60c56870ee4dbb663aa0dc8c8040acd186de25a8a6b35293bdf08858461c"
Sep 17 03:33:50.663053 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.663020    2576 scope.go:117] "RemoveContainer" containerID="7c9e60c56870ee4dbb663aa0dc8c8040acd186de25a8a6b35293bdf08858461c"
Sep 17 03:33:50.663580 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:33:50.663547    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c9e60c56870ee4dbb663aa0dc8c8040acd186de25a8a6b35293bdf08858461c\": container with ID starting with 7c9e60c56870ee4dbb663aa0dc8c8040acd186de25a8a6b35293bdf08858461c not found: ID does not exist" containerID="7c9e60c56870ee4dbb663aa0dc8c8040acd186de25a8a6b35293bdf08858461c"
Sep 17 03:33:50.663692 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.663587    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9e60c56870ee4dbb663aa0dc8c8040acd186de25a8a6b35293bdf08858461c"} err="failed to get container status \"7c9e60c56870ee4dbb663aa0dc8c8040acd186de25a8a6b35293bdf08858461c\": rpc error: code = NotFound desc = could not find container \"7c9e60c56870ee4dbb663aa0dc8c8040acd186de25a8a6b35293bdf08858461c\": container with ID starting with 7c9e60c56870ee4dbb663aa0dc8c8040acd186de25a8a6b35293bdf08858461c not found: ID does not exist"
Sep 17 03:33:50.733445 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.733399    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41a5703d-e221-4a31-ab30-f734e9f5b04a-plugins-config-volume\") pod \"41a5703d-e221-4a31-ab30-f734e9f5b04a\" (UID: \"41a5703d-e221-4a31-ab30-f734e9f5b04a\") "
Sep 17 03:33:50.733618 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.733459    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hn2qp\" (UniqueName: \"kubernetes.io/projected/41a5703d-e221-4a31-ab30-f734e9f5b04a-kube-api-access-hn2qp\") pod \"41a5703d-e221-4a31-ab30-f734e9f5b04a\" (UID: \"41a5703d-e221-4a31-ab30-f734e9f5b04a\") "
Sep 17 03:33:50.733807 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.733784    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41a5703d-e221-4a31-ab30-f734e9f5b04a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "41a5703d-e221-4a31-ab30-f734e9f5b04a" (UID: "41a5703d-e221-4a31-ab30-f734e9f5b04a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:33:50.735563 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.735533    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a5703d-e221-4a31-ab30-f734e9f5b04a-kube-api-access-hn2qp" (OuterVolumeSpecName: "kube-api-access-hn2qp") pod "41a5703d-e221-4a31-ab30-f734e9f5b04a" (UID: "41a5703d-e221-4a31-ab30-f734e9f5b04a"). InnerVolumeSpecName "kube-api-access-hn2qp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:33:50.834633 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.834532    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41a5703d-e221-4a31-ab30-f734e9f5b04a-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:33:50.834633 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.834567    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn2qp\" (UniqueName: \"kubernetes.io/projected/41a5703d-e221-4a31-ab30-f734e9f5b04a-kube-api-access-hn2qp\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:33:50.976192 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.976143    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"]
Sep 17 03:33:50.985192 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:50.985162    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3b91ce2e-68484d5ddb-zxz7d"]
Sep 17 03:33:51.094291 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:51.094206    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:51.689216 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:51.689181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:51.821381 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:51.821348    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:52.169734 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:52.169700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:52.300794 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:52.300762    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41a5703d-e221-4a31-ab30-f734e9f5b04a" path="/var/lib/kubelet/pods/41a5703d-e221-4a31-ab30-f734e9f5b04a/volumes"
Sep 17 03:33:52.447478 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:52.447384    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:52.607109 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:52.607071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:52.939979 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:52.939944    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:53.652682 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:53.652649    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:54.367839 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.367803    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"]
Sep 17 03:33:54.368239 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.368226    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41a5703d-e221-4a31-ab30-f734e9f5b04a" containerName="payload-pre-processing"
Sep 17 03:33:54.368239 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.368240    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="41a5703d-e221-4a31-ab30-f734e9f5b04a" containerName="payload-pre-processing"
Sep 17 03:33:54.368315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.368297    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="41a5703d-e221-4a31-ab30-f734e9f5b04a" containerName="payload-pre-processing"
Sep 17 03:33:54.370977 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.370960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.373664 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.373636    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-297324\"/\"default-dockercfg-8ckzw\""
Sep 17 03:33:54.373664 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.373663    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-297324\"/\"test-model-297324-kserve-self-signed-certs\""
Sep 17 03:33:54.373860 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.373686    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-297324\"/\"openshift-service-ca.crt\""
Sep 17 03:33:54.373860 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.373637    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-297324\"/\"kube-root-ca.crt\""
Sep 17 03:33:54.381211 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.381185    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"]
Sep 17 03:33:54.468803 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.468761    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t9mx\" (UniqueName: \"kubernetes.io/projected/4a14a289-96db-4bfd-943a-e3863008ffda-kube-api-access-8t9mx\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.468803 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.468802    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-model-cache\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.469021 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.468826    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a14a289-96db-4bfd-943a-e3863008ffda-tls-certs\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.469021 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.468895    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-dshm\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.469021 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.468938    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-home\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.469021 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.468953    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-tmp-dir\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.482350 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.482317    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:54.569896 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.569862    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-model-cache\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.570057 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.569905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a14a289-96db-4bfd-943a-e3863008ffda-tls-certs\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.570057 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.570019    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-dshm\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.570057 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.570044    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-home\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.570209 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.570064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-tmp-dir\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.570265 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.570209    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9mx\" (UniqueName: \"kubernetes.io/projected/4a14a289-96db-4bfd-943a-e3863008ffda-kube-api-access-8t9mx\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.570324 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.570279    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-model-cache\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.570451 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.570414    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-home\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.570626 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.570609    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-tmp-dir\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.572150 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.572130    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-dshm\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.572460 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.572417    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a14a289-96db-4bfd-943a-e3863008ffda-tls-certs\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.577939 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.577921    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t9mx\" (UniqueName: \"kubernetes.io/projected/4a14a289-96db-4bfd-943a-e3863008ffda-kube-api-access-8t9mx\") pod \"test-model-297324-kserve-7fd46548c-rdj4g\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") " pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.582607 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.582585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:54.681062 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.680979    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:33:54.856215 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.856180    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:54.955960 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.955859    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"]
Sep 17 03:33:54.956947 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:54.956910    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:54.959457 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:33:54.959417    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a14a289_96db_4bfd_943a_e3863008ffda.slice/crio-12c1ba6410455ab5ac41325f1529cf317d2db51fe2ebc1ad1b77e9b4d42f0324 WatchSource:0}: Error finding container 12c1ba6410455ab5ac41325f1529cf317d2db51fe2ebc1ad1b77e9b4d42f0324: Status 404 returned error can't find the container with id 12c1ba6410455ab5ac41325f1529cf317d2db51fe2ebc1ad1b77e9b4d42f0324
Sep 17 03:33:55.421715 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:55.421683    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:55.686511 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:55.686394    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g" event={"ID":"4a14a289-96db-4bfd-943a-e3863008ffda","Type":"ContainerStarted","Data":"b0b6d699cbfa131c3063f48a820656692a598d00a24d7daf29cdde632c703cae"}
Sep 17 03:33:55.686511 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:55.686467    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g" event={"ID":"4a14a289-96db-4bfd-943a-e3863008ffda","Type":"ContainerStarted","Data":"12c1ba6410455ab5ac41325f1529cf317d2db51fe2ebc1ad1b77e9b4d42f0324"}
Sep 17 03:33:55.705876 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:55.705818    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g" podStartSLOduration=1.705801196 podStartE2EDuration="1.705801196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:33:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:33:55.702719883 +0000 UTC m=+864.047445962" watchObservedRunningTime="2026-09-17 03:33:55.705801196 +0000 UTC m=+864.050527307"
Sep 17 03:33:56.224801 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:56.224763    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:56.324471 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:56.324414    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:56.569474 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:56.569410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:56.940880 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:56.940794    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:57.040731 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:57.040696    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:57.408966 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:57.408921    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:58.298293 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:58.298266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:59.106942 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:59.106899    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:33:59.538000 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:59.537902    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:33:59.807264 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:33:59.807232    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:00.563871 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:00.563838    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:03.527716 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:03.527683    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:04.552583 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:04.552541    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:04.681680 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:04.681645    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:34:04.681902 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:04.681703    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:34:04.695241 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:04.695214    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:34:04.748607 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:04.748572    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:34:05.434915 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:05.434876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:06.138334 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:06.138303    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:06.881175 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:06.881145    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:07.114156 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:07.114116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:07.812689 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:07.812654    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:08.562089 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:08.562051    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:09.518363 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:09.518326    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:10.408097 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:10.408016    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:11.099665 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:11.099629    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:11.822093 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:11.822058    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:12.699921 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:12.699887    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:13.510050 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:13.510012    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:14.195595 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:14.195563    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:14.967273 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:14.967245    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:14.994897 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:14.994867    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"]
Sep 17 03:34:14.995138 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:14.995111    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9" podUID="ad3a4de5-016e-4816-85e4-feb75d050596" containerName="payload-processing" containerID="cri-o://00a9bdded79097f0f8f23de787d009c155836d4db2acc74b60341425ddfd120f" gracePeriod=30
Sep 17 03:34:15.068503 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.068461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:15.167269 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.167240    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:15.267110 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.267068    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:15.273327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.273296    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"
Sep 17 03:34:15.367722 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.367681    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad3a4de5-016e-4816-85e4-feb75d050596-plugins-config-volume\") pod \"ad3a4de5-016e-4816-85e4-feb75d050596\" (UID: \"ad3a4de5-016e-4816-85e4-feb75d050596\") "
Sep 17 03:34:15.367923 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.367782    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwnht\" (UniqueName: \"kubernetes.io/projected/ad3a4de5-016e-4816-85e4-feb75d050596-kube-api-access-rwnht\") pod \"ad3a4de5-016e-4816-85e4-feb75d050596\" (UID: \"ad3a4de5-016e-4816-85e4-feb75d050596\") "
Sep 17 03:34:15.368161 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.368116    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad3a4de5-016e-4816-85e4-feb75d050596-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ad3a4de5-016e-4816-85e4-feb75d050596" (UID: "ad3a4de5-016e-4816-85e4-feb75d050596"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:34:15.370196 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.370169    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad3a4de5-016e-4816-85e4-feb75d050596-kube-api-access-rwnht" (OuterVolumeSpecName: "kube-api-access-rwnht") pod "ad3a4de5-016e-4816-85e4-feb75d050596" (UID: "ad3a4de5-016e-4816-85e4-feb75d050596"). InnerVolumeSpecName "kube-api-access-rwnht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:34:15.416195 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.416158    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:15.468459 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.468415    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad3a4de5-016e-4816-85e4-feb75d050596-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:34:15.468459 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.468455    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwnht\" (UniqueName: \"kubernetes.io/projected/ad3a4de5-016e-4816-85e4-feb75d050596-kube-api-access-rwnht\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:34:15.636055 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.636018    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:15.801756 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.801722    2576 generic.go:356] "Generic (PLEG): container finished" podID="ad3a4de5-016e-4816-85e4-feb75d050596" containerID="00a9bdded79097f0f8f23de787d009c155836d4db2acc74b60341425ddfd120f" exitCode=0
Sep 17 03:34:15.801924 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.801754    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9" event={"ID":"ad3a4de5-016e-4816-85e4-feb75d050596","Type":"ContainerDied","Data":"00a9bdded79097f0f8f23de787d009c155836d4db2acc74b60341425ddfd120f"}
Sep 17 03:34:15.801924 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.801807    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9" event={"ID":"ad3a4de5-016e-4816-85e4-feb75d050596","Type":"ContainerDied","Data":"be7ea1a65e4186a953d42d4ef730d3407e6d92ae620d5c87b2c4acc592d7ec51"}
Sep 17 03:34:15.801924 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.801829    2576 scope.go:117] "RemoveContainer" containerID="00a9bdded79097f0f8f23de787d009c155836d4db2acc74b60341425ddfd120f"
Sep 17 03:34:15.801924 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.801761    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"
Sep 17 03:34:15.811122 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.811104    2576 scope.go:117] "RemoveContainer" containerID="00a9bdded79097f0f8f23de787d009c155836d4db2acc74b60341425ddfd120f"
Sep 17 03:34:15.811413 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:34:15.811395    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00a9bdded79097f0f8f23de787d009c155836d4db2acc74b60341425ddfd120f\": container with ID starting with 00a9bdded79097f0f8f23de787d009c155836d4db2acc74b60341425ddfd120f not found: ID does not exist" containerID="00a9bdded79097f0f8f23de787d009c155836d4db2acc74b60341425ddfd120f"
Sep 17 03:34:15.811492 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.811423    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00a9bdded79097f0f8f23de787d009c155836d4db2acc74b60341425ddfd120f"} err="failed to get container status \"00a9bdded79097f0f8f23de787d009c155836d4db2acc74b60341425ddfd120f\": rpc error: code = NotFound desc = could not find container \"00a9bdded79097f0f8f23de787d009c155836d4db2acc74b60341425ddfd120f\": container with ID starting with 00a9bdded79097f0f8f23de787d009c155836d4db2acc74b60341425ddfd120f not found: ID does not exist"
Sep 17 03:34:15.824028 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.823998    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"]
Sep 17 03:34:15.827276 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:15.827254    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f6a774-5b74bbf4ff-t5lw9"]
Sep 17 03:34:16.215938 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:16.215899    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:16.294691 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:16.294657    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad3a4de5-016e-4816-85e4-feb75d050596" path="/var/lib/kubelet/pods/ad3a4de5-016e-4816-85e4-feb75d050596/volumes"
Sep 17 03:34:17.232947 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:17.232911    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:17.954059 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:17.954029    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:18.927421 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:18.927383    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:19.717038 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:19.717006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:20.488960 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:20.488926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:20.715769 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:20.715729    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:23.436634 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:23.436590    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:34:25.091440 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:25.091335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:25.164094 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:25.164063    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:25.224021 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:25.223986    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:25.299370 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:25.299332    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:25.370007 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:25.369899    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:25.448620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:25.448587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:25.825642 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:25.825609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:26.125553 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:26.125468    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:26.224642 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:26.224602    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:26.324904 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:26.324851    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:26.722069 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:26.722035    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:27.627221 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:27.627183    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:27.721716 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:27.721669    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:27.867671 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:27.867638    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:28.098499 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:28.098459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:28.436249 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:28.436143    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:28.532967 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:28.532929    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:28.633710 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:28.633682    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:28.825390 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:28.825354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:29.377025 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:29.376992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:29.479656 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:29.479622    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:31.553124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:31.553078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:31.707779 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:31.707737    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:31.801574 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:31.801535    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:32.223049 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:32.223020    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:34:32.223217 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:32.223024    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:34:32.383228 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:32.383199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:33.155754 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:33.155717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:34.035610 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:34.035574    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:34.651595 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:34.651553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:34.842852 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:34.842818    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:34.961547 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:34.961458    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:35.109345 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.109301    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:35.378136 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.378107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:35.472147 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.472114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:35.528697 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.528661    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"]
Sep 17 03:34:35.529075 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.529062    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad3a4de5-016e-4816-85e4-feb75d050596" containerName="payload-processing"
Sep 17 03:34:35.529143 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.529077    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad3a4de5-016e-4816-85e4-feb75d050596" containerName="payload-processing"
Sep 17 03:34:35.529180 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.529147    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad3a4de5-016e-4816-85e4-feb75d050596" containerName="payload-processing"
Sep 17 03:34:35.533835 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.533810    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"
Sep 17 03:34:35.536298 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.536273    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-5062bf-dockercfg-lhdsx\""
Sep 17 03:34:35.536656 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.536628    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-5062bf\""
Sep 17 03:34:35.542827 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.542804    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"]
Sep 17 03:34:35.572950 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.572917    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:35.651631 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.651545    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s55pj\" (UniqueName: \"kubernetes.io/projected/6587a5cf-2c36-4129-88d3-640381b215f0-kube-api-access-s55pj\") pod \"payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw\" (UID: \"6587a5cf-2c36-4129-88d3-640381b215f0\") " pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"
Sep 17 03:34:35.651631 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.651585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6587a5cf-2c36-4129-88d3-640381b215f0-plugins-config-volume\") pod \"payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw\" (UID: \"6587a5cf-2c36-4129-88d3-640381b215f0\") " pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"
Sep 17 03:34:35.752811 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.752773    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s55pj\" (UniqueName: \"kubernetes.io/projected/6587a5cf-2c36-4129-88d3-640381b215f0-kube-api-access-s55pj\") pod \"payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw\" (UID: \"6587a5cf-2c36-4129-88d3-640381b215f0\") " pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"
Sep 17 03:34:35.752811 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.752817    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6587a5cf-2c36-4129-88d3-640381b215f0-plugins-config-volume\") pod \"payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw\" (UID: \"6587a5cf-2c36-4129-88d3-640381b215f0\") " pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"
Sep 17 03:34:35.753615 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.753597    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6587a5cf-2c36-4129-88d3-640381b215f0-plugins-config-volume\") pod \"payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw\" (UID: \"6587a5cf-2c36-4129-88d3-640381b215f0\") " pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"
Sep 17 03:34:35.761252 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.761221    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s55pj\" (UniqueName: \"kubernetes.io/projected/6587a5cf-2c36-4129-88d3-640381b215f0-kube-api-access-s55pj\") pod \"payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw\" (UID: \"6587a5cf-2c36-4129-88d3-640381b215f0\") " pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"
Sep 17 03:34:35.845733 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:35.845690    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"
Sep 17 03:34:36.125132 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:36.125106    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"]
Sep 17 03:34:36.126624 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:34:36.126595    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6587a5cf_2c36_4129_88d3_640381b215f0.slice/crio-71a4a4620333b58f6f4f0e978cd8f1d0bf3685e536432faa49ad98c0a205836b WatchSource:0}: Error finding container 71a4a4620333b58f6f4f0e978cd8f1d0bf3685e536432faa49ad98c0a205836b: Status 404 returned error can't find the container with id 71a4a4620333b58f6f4f0e978cd8f1d0bf3685e536432faa49ad98c0a205836b
Sep 17 03:34:36.923969 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:36.923922    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw" event={"ID":"6587a5cf-2c36-4129-88d3-640381b215f0","Type":"ContainerStarted","Data":"38a13f3d45be44f2fb1fb6151a3cd0eaf5c6ba149c9ce714cdd97c9db1f0d4ed"}
Sep 17 03:34:36.924351 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:36.924119    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"
Sep 17 03:34:36.924351 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:36.924155    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw" event={"ID":"6587a5cf-2c36-4129-88d3-640381b215f0","Type":"ContainerStarted","Data":"71a4a4620333b58f6f4f0e978cd8f1d0bf3685e536432faa49ad98c0a205836b"}
Sep 17 03:34:36.941223 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:36.941159    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw" podStartSLOduration=1.941143055 podStartE2EDuration="1.941143055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:34:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:34:36.939740983 +0000 UTC m=+905.284467064" watchObservedRunningTime="2026-09-17 03:34:36.941143055 +0000 UTC m=+905.285869133"
Sep 17 03:34:40.078011 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:40.077967    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:42.652932 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:42.652893    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:34:47.931578 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:47.931545    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"
Sep 17 03:34:58.539626 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:58.539587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:34:59.313073 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:34:59.313034    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:00.120938 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:00.120894    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:00.948383 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:00.948347    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:01.301774 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:01.301736    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:01.723488 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:01.723390    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:01.825701 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:01.825665    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:02.506835 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:02.506791    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:06.945699 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:06.945657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:07.059181 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:07.059146    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:07.146993 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:07.146960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:07.420089 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:07.420054    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:13.549463 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:13.549410    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:35:22.512501 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:22.512468    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:22.995360 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:22.995323    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:29.132073 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:29.132031    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:29.197641 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:29.197607    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:29.294155 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:29.294116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:29.394019 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:29.393931    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:30.550771 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:30.550736    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:35:36.720890 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:36.720855    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:39.817493 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:39.817459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:40.571076 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:40.571038    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:47.128401 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:47.128366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:47.247653 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:47.247618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:47.340647 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:47.340606    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:47.441613 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:47.441518    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:48.064712 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:48.064672    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:48.172659 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:48.172622    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:48.303816 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:48.303778    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:49.676367 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:49.676336    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:55.241860 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:55.241776    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:35:55.559625 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:55.559592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:55.628105 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:55.628067    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:35:56.236078 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:35:56.236045    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:02.220173 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:02.220134    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:02.319387 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:02.319357    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:02.469126 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:02.469091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:02.840723 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:02.840686    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:03.544951 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:03.544915    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:08.498149 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:08.498109    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:09.125691 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:09.125656    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:09.218114 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:09.218081    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:09.517578 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:09.517491    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:10.428501 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:10.428470    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:11.107344 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:11.107306    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:11.807447 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:11.807394    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:11.909011 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:11.908973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:12.009735 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:12.009703    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:12.543937 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:12.543905    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:13.349639 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:13.349599    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:13.488710 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:13.488678    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:13.613981 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:13.613882    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:14.033099 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.033008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:14.135141 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.135108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:14.183985 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.183950    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"]
Sep 17 03:36:14.187527 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.187508    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"
Sep 17 03:36:14.189750 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.189730    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-cdaa7ee2\""
Sep 17 03:36:14.189847 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.189732    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-cdaa7ee2-dockercfg-fd64v\""
Sep 17 03:36:14.199514 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.199485    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"]
Sep 17 03:36:14.230543 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.230508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrh92\" (UniqueName: \"kubernetes.io/projected/440f5e53-efda-4289-b0f4-32587e1d77aa-kube-api-access-wrh92\") pod \"payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c\" (UID: \"440f5e53-efda-4289-b0f4-32587e1d77aa\") " pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"
Sep 17 03:36:14.230720 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.230565    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440f5e53-efda-4289-b0f4-32587e1d77aa-plugins-config-volume\") pod \"payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c\" (UID: \"440f5e53-efda-4289-b0f4-32587e1d77aa\") " pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"
Sep 17 03:36:14.331620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.331584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440f5e53-efda-4289-b0f4-32587e1d77aa-plugins-config-volume\") pod \"payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c\" (UID: \"440f5e53-efda-4289-b0f4-32587e1d77aa\") " pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"
Sep 17 03:36:14.331814 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.331691    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrh92\" (UniqueName: \"kubernetes.io/projected/440f5e53-efda-4289-b0f4-32587e1d77aa-kube-api-access-wrh92\") pod \"payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c\" (UID: \"440f5e53-efda-4289-b0f4-32587e1d77aa\") " pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"
Sep 17 03:36:14.332289 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.332269    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440f5e53-efda-4289-b0f4-32587e1d77aa-plugins-config-volume\") pod \"payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c\" (UID: \"440f5e53-efda-4289-b0f4-32587e1d77aa\") " pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"
Sep 17 03:36:14.340656 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.340625    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrh92\" (UniqueName: \"kubernetes.io/projected/440f5e53-efda-4289-b0f4-32587e1d77aa-kube-api-access-wrh92\") pod \"payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c\" (UID: \"440f5e53-efda-4289-b0f4-32587e1d77aa\") " pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"
Sep 17 03:36:14.486058 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.486026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:14.498338 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.498314    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"
Sep 17 03:36:14.786838 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:36:14.786804    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod440f5e53_efda_4289_b0f4_32587e1d77aa.slice/crio-9701d163d409c09ab80e6881b895a0fde002c4374d7babe296d8ee6036fedd99 WatchSource:0}: Error finding container 9701d163d409c09ab80e6881b895a0fde002c4374d7babe296d8ee6036fedd99: Status 404 returned error can't find the container with id 9701d163d409c09ab80e6881b895a0fde002c4374d7babe296d8ee6036fedd99
Sep 17 03:36:14.786838 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:14.786830    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"]
Sep 17 03:36:15.313821 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:15.313786    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:15.465809 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:15.465770    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c" event={"ID":"440f5e53-efda-4289-b0f4-32587e1d77aa","Type":"ContainerStarted","Data":"59759ad2c9c136adabdf26af06644f002827f7a7beac2c884bcbd1dc3e197208"}
Sep 17 03:36:15.465985 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:15.465837    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c" event={"ID":"440f5e53-efda-4289-b0f4-32587e1d77aa","Type":"ContainerStarted","Data":"9701d163d409c09ab80e6881b895a0fde002c4374d7babe296d8ee6036fedd99"}
Sep 17 03:36:15.465985 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:15.465868    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"
Sep 17 03:36:15.489073 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:15.489022    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c" podStartSLOduration=1.489004534 podStartE2EDuration="1.489004534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:36:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:36:15.488349907 +0000 UTC m=+1003.833075985" watchObservedRunningTime="2026-09-17 03:36:15.489004534 +0000 UTC m=+1003.833730613"
Sep 17 03:36:18.807810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:18.807761    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:19.405117 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.405084    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:19.435862 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.435757    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"]
Sep 17 03:36:19.436175 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.436136    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw" podUID="6587a5cf-2c36-4129-88d3-640381b215f0" containerName="payload-processing" containerID="cri-o://38a13f3d45be44f2fb1fb6151a3cd0eaf5c6ba149c9ce714cdd97c9db1f0d4ed" gracePeriod=30
Sep 17 03:36:19.505425 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.505389    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:19.600358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.600316    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:19.658868 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.658775    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:19.703855 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.703824    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"
Sep 17 03:36:19.785749 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.785718    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6587a5cf-2c36-4129-88d3-640381b215f0-plugins-config-volume\") pod \"6587a5cf-2c36-4129-88d3-640381b215f0\" (UID: \"6587a5cf-2c36-4129-88d3-640381b215f0\") "
Sep 17 03:36:19.785931 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.785783    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s55pj\" (UniqueName: \"kubernetes.io/projected/6587a5cf-2c36-4129-88d3-640381b215f0-kube-api-access-s55pj\") pod \"6587a5cf-2c36-4129-88d3-640381b215f0\" (UID: \"6587a5cf-2c36-4129-88d3-640381b215f0\") "
Sep 17 03:36:19.786175 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.786149    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6587a5cf-2c36-4129-88d3-640381b215f0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6587a5cf-2c36-4129-88d3-640381b215f0" (UID: "6587a5cf-2c36-4129-88d3-640381b215f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:36:19.787963 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.787941    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6587a5cf-2c36-4129-88d3-640381b215f0-kube-api-access-s55pj" (OuterVolumeSpecName: "kube-api-access-s55pj") pod "6587a5cf-2c36-4129-88d3-640381b215f0" (UID: "6587a5cf-2c36-4129-88d3-640381b215f0"). InnerVolumeSpecName "kube-api-access-s55pj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:36:19.887477 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.887442    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s55pj\" (UniqueName: \"kubernetes.io/projected/6587a5cf-2c36-4129-88d3-640381b215f0-kube-api-access-s55pj\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:36:19.887477 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.887474    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6587a5cf-2c36-4129-88d3-640381b215f0-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:36:19.972022 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:19.971912    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:20.073276 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:20.073239    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:20.498487 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:20.498450    2576 generic.go:356] "Generic (PLEG): container finished" podID="6587a5cf-2c36-4129-88d3-640381b215f0" containerID="38a13f3d45be44f2fb1fb6151a3cd0eaf5c6ba149c9ce714cdd97c9db1f0d4ed" exitCode=0
Sep 17 03:36:20.498656 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:20.498498    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"
Sep 17 03:36:20.498656 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:20.498522    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw" event={"ID":"6587a5cf-2c36-4129-88d3-640381b215f0","Type":"ContainerDied","Data":"38a13f3d45be44f2fb1fb6151a3cd0eaf5c6ba149c9ce714cdd97c9db1f0d4ed"}
Sep 17 03:36:20.498656 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:20.498579    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw" event={"ID":"6587a5cf-2c36-4129-88d3-640381b215f0","Type":"ContainerDied","Data":"71a4a4620333b58f6f4f0e978cd8f1d0bf3685e536432faa49ad98c0a205836b"}
Sep 17 03:36:20.498656 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:20.498607    2576 scope.go:117] "RemoveContainer" containerID="38a13f3d45be44f2fb1fb6151a3cd0eaf5c6ba149c9ce714cdd97c9db1f0d4ed"
Sep 17 03:36:20.507819 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:20.507800    2576 scope.go:117] "RemoveContainer" containerID="38a13f3d45be44f2fb1fb6151a3cd0eaf5c6ba149c9ce714cdd97c9db1f0d4ed"
Sep 17 03:36:20.508117 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:36:20.508094    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38a13f3d45be44f2fb1fb6151a3cd0eaf5c6ba149c9ce714cdd97c9db1f0d4ed\": container with ID starting with 38a13f3d45be44f2fb1fb6151a3cd0eaf5c6ba149c9ce714cdd97c9db1f0d4ed not found: ID does not exist" containerID="38a13f3d45be44f2fb1fb6151a3cd0eaf5c6ba149c9ce714cdd97c9db1f0d4ed"
Sep 17 03:36:20.508222 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:20.508124    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a13f3d45be44f2fb1fb6151a3cd0eaf5c6ba149c9ce714cdd97c9db1f0d4ed"} err="failed to get container status \"38a13f3d45be44f2fb1fb6151a3cd0eaf5c6ba149c9ce714cdd97c9db1f0d4ed\": rpc error: code = NotFound desc = could not find container \"38a13f3d45be44f2fb1fb6151a3cd0eaf5c6ba149c9ce714cdd97c9db1f0d4ed\": container with ID starting with 38a13f3d45be44f2fb1fb6151a3cd0eaf5c6ba149c9ce714cdd97c9db1f0d4ed not found: ID does not exist"
Sep 17 03:36:20.515828 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:20.515801    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"]
Sep 17 03:36:20.518194 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:20.518169    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-5062bf-785b8bf78f-hw7rw"]
Sep 17 03:36:20.828004 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:20.827949    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:21.488972 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:21.488937    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:22.295341 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:22.295309    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6587a5cf-2c36-4129-88d3-640381b215f0" path="/var/lib/kubelet/pods/6587a5cf-2c36-4129-88d3-640381b215f0/volumes"
Sep 17 03:36:22.391481 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:22.391424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:23.118062 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:23.118031    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:23.783908 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:23.783848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:23.918907 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:23.918860    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:24.455334 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:24.455294    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:24.555308 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:24.555270    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:24.735211 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:24.735125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:24.870406 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:24.870354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:25.073745 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:25.073697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:25.537232 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:25.537145    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:26.408017 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:26.407984    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:26.474401 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:26.474371    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"
Sep 17 03:36:26.521921 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:26.521881    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:27.399806 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:27.399769    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:27.941249 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:27.941211    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:36:29.192400 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:29.192349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:32.471104 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:32.471070    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:32.586158 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:32.586123    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:32.860489 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:32.860397    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd"]
Sep 17 03:36:32.860801 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:32.860788    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6587a5cf-2c36-4129-88d3-640381b215f0" containerName="payload-processing"
Sep 17 03:36:32.860846 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:32.860803    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6587a5cf-2c36-4129-88d3-640381b215f0" containerName="payload-processing"
Sep 17 03:36:32.860881 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:32.860872    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6587a5cf-2c36-4129-88d3-640381b215f0" containerName="payload-processing"
Sep 17 03:36:32.865003 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:32.864982    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd"
Sep 17 03:36:32.868345 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:32.868323    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8vwnw\""
Sep 17 03:36:32.881242 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:32.881209    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd"]
Sep 17 03:36:33.009307 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:33.009274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37ebb122-6f00-4efc-af79-259ed38ab82b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd\" (UID: \"37ebb122-6f00-4efc-af79-259ed38ab82b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd"
Sep 17 03:36:33.110250 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:33.110205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37ebb122-6f00-4efc-af79-259ed38ab82b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd\" (UID: \"37ebb122-6f00-4efc-af79-259ed38ab82b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd"
Sep 17 03:36:33.110926 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:33.110864    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37ebb122-6f00-4efc-af79-259ed38ab82b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd\" (UID: \"37ebb122-6f00-4efc-af79-259ed38ab82b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd"
Sep 17 03:36:33.175218 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:33.175188    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd"
Sep 17 03:36:33.243037 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:33.243008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:33.467151 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:33.467121    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd"]
Sep 17 03:36:33.467923 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:36:33.467894    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ebb122_6f00_4efc_af79_259ed38ab82b.slice/crio-9aedc19fe2447f821116e537122cb88f7fd93b564725550f81e1097c543ed166 WatchSource:0}: Error finding container 9aedc19fe2447f821116e537122cb88f7fd93b564725550f81e1097c543ed166: Status 404 returned error can't find the container with id 9aedc19fe2447f821116e537122cb88f7fd93b564725550f81e1097c543ed166
Sep 17 03:36:33.470114 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:33.470094    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 03:36:33.574572 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:33.574537    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd" event={"ID":"37ebb122-6f00-4efc-af79-259ed38ab82b","Type":"ContainerStarted","Data":"9aedc19fe2447f821116e537122cb88f7fd93b564725550f81e1097c543ed166"}
Sep 17 03:36:34.567851 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:34.567788    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:35.490402 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:35.490363    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:36.600339 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:36.600299    2576 generic.go:356] "Generic (PLEG): container finished" podID="37ebb122-6f00-4efc-af79-259ed38ab82b" containerID="c05ca5635984049437b5b2aa46a205af6180820ede80bfcca427cf2056a5f1e3" exitCode=0
Sep 17 03:36:36.600743 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:36.600357    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd" event={"ID":"37ebb122-6f00-4efc-af79-259ed38ab82b","Type":"ContainerDied","Data":"c05ca5635984049437b5b2aa46a205af6180820ede80bfcca427cf2056a5f1e3"}
Sep 17 03:36:36.604175 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:36.604143    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:37.350020 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:37.349985    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:37.751273 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:37.751242    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd"
Sep 17 03:36:37.854191 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:37.854153    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37ebb122-6f00-4efc-af79-259ed38ab82b-maas-api-service-ca\") pod \"37ebb122-6f00-4efc-af79-259ed38ab82b\" (UID: \"37ebb122-6f00-4efc-af79-259ed38ab82b\") "
Sep 17 03:36:37.854609 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:37.854578    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37ebb122-6f00-4efc-af79-259ed38ab82b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "37ebb122-6f00-4efc-af79-259ed38ab82b" (UID: "37ebb122-6f00-4efc-af79-259ed38ab82b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:36:37.955627 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:37.955532    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37ebb122-6f00-4efc-af79-259ed38ab82b-maas-api-service-ca\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:36:38.217682 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:38.217584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:38.613211 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:38.613173    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd"
Sep 17 03:36:38.613211 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:38.613201    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd" event={"ID":"37ebb122-6f00-4efc-af79-259ed38ab82b","Type":"ContainerDied","Data":"9aedc19fe2447f821116e537122cb88f7fd93b564725550f81e1097c543ed166"}
Sep 17 03:36:38.613538 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:38.613265    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9aedc19fe2447f821116e537122cb88f7fd93b564725550f81e1097c543ed166"
Sep 17 03:36:38.906740 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:38.906657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:40.992610 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:40.992557    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:41.020200 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.020166    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"]
Sep 17 03:36:41.020416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.020388    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c" podUID="440f5e53-efda-4289-b0f4-32587e1d77aa" containerName="payload-processing" containerID="cri-o://59759ad2c9c136adabdf26af06644f002827f7a7beac2c884bcbd1dc3e197208" gracePeriod=30
Sep 17 03:36:41.065719 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.065668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:41.127646 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.127613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:41.215142 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.215103    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd"]
Sep 17 03:36:41.216282 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.216248    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:41.217662 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.217636    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cdaa7ee2-klmfd"]
Sep 17 03:36:41.274999 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.274975    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"
Sep 17 03:36:41.278124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.278091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:41.390784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.390745    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440f5e53-efda-4289-b0f4-32587e1d77aa-plugins-config-volume\") pod \"440f5e53-efda-4289-b0f4-32587e1d77aa\" (UID: \"440f5e53-efda-4289-b0f4-32587e1d77aa\") "
Sep 17 03:36:41.390976 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.390797    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrh92\" (UniqueName: \"kubernetes.io/projected/440f5e53-efda-4289-b0f4-32587e1d77aa-kube-api-access-wrh92\") pod \"440f5e53-efda-4289-b0f4-32587e1d77aa\" (UID: \"440f5e53-efda-4289-b0f4-32587e1d77aa\") "
Sep 17 03:36:41.391240 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.391197    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/440f5e53-efda-4289-b0f4-32587e1d77aa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "440f5e53-efda-4289-b0f4-32587e1d77aa" (UID: "440f5e53-efda-4289-b0f4-32587e1d77aa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:36:41.391365 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.391326    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440f5e53-efda-4289-b0f4-32587e1d77aa-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:36:41.393825 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.393727    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/440f5e53-efda-4289-b0f4-32587e1d77aa-kube-api-access-wrh92" (OuterVolumeSpecName: "kube-api-access-wrh92") pod "440f5e53-efda-4289-b0f4-32587e1d77aa" (UID: "440f5e53-efda-4289-b0f4-32587e1d77aa"). InnerVolumeSpecName "kube-api-access-wrh92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:36:41.492374 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.492336    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrh92\" (UniqueName: \"kubernetes.io/projected/440f5e53-efda-4289-b0f4-32587e1d77aa-kube-api-access-wrh92\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:36:41.584773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.584735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:41.631262 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.631225    2576 generic.go:356] "Generic (PLEG): container finished" podID="440f5e53-efda-4289-b0f4-32587e1d77aa" containerID="59759ad2c9c136adabdf26af06644f002827f7a7beac2c884bcbd1dc3e197208" exitCode=0
Sep 17 03:36:41.631476 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.631268    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"
Sep 17 03:36:41.631476 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.631315    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c" event={"ID":"440f5e53-efda-4289-b0f4-32587e1d77aa","Type":"ContainerDied","Data":"59759ad2c9c136adabdf26af06644f002827f7a7beac2c884bcbd1dc3e197208"}
Sep 17 03:36:41.631476 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.631367    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c" event={"ID":"440f5e53-efda-4289-b0f4-32587e1d77aa","Type":"ContainerDied","Data":"9701d163d409c09ab80e6881b895a0fde002c4374d7babe296d8ee6036fedd99"}
Sep 17 03:36:41.631476 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.631392    2576 scope.go:117] "RemoveContainer" containerID="59759ad2c9c136adabdf26af06644f002827f7a7beac2c884bcbd1dc3e197208"
Sep 17 03:36:41.641068 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.641033    2576 scope.go:117] "RemoveContainer" containerID="59759ad2c9c136adabdf26af06644f002827f7a7beac2c884bcbd1dc3e197208"
Sep 17 03:36:41.641383 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:36:41.641362    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59759ad2c9c136adabdf26af06644f002827f7a7beac2c884bcbd1dc3e197208\": container with ID starting with 59759ad2c9c136adabdf26af06644f002827f7a7beac2c884bcbd1dc3e197208 not found: ID does not exist" containerID="59759ad2c9c136adabdf26af06644f002827f7a7beac2c884bcbd1dc3e197208"
Sep 17 03:36:41.641468 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.641390    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59759ad2c9c136adabdf26af06644f002827f7a7beac2c884bcbd1dc3e197208"} err="failed to get container status \"59759ad2c9c136adabdf26af06644f002827f7a7beac2c884bcbd1dc3e197208\": rpc error: code = NotFound desc = could not find container \"59759ad2c9c136adabdf26af06644f002827f7a7beac2c884bcbd1dc3e197208\": container with ID starting with 59759ad2c9c136adabdf26af06644f002827f7a7beac2c884bcbd1dc3e197208 not found: ID does not exist"
Sep 17 03:36:41.652094 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.652062    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"]
Sep 17 03:36:41.656982 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.656953    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:41.657802 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:41.657782    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cdaa7ee2-5c68b8c6cc-8nf8c"]
Sep 17 03:36:42.294386 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:42.294354    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37ebb122-6f00-4efc-af79-259ed38ab82b" path="/var/lib/kubelet/pods/37ebb122-6f00-4efc-af79-259ed38ab82b/volumes"
Sep 17 03:36:42.294774 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:42.294732    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="440f5e53-efda-4289-b0f4-32587e1d77aa" path="/var/lib/kubelet/pods/440f5e53-efda-4289-b0f4-32587e1d77aa/volumes"
Sep 17 03:36:42.361954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:42.361915    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:43.099016 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:43.098984    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:44.042629 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:44.042597    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:44.764113 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:44.764074    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:47.767811 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:47.767781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:47.826733 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:47.826702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:47.951459 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:47.951404    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:48.010755 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:48.010725    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:48.333970 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:48.333934    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:48.434706 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:48.434676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:48.539627 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:48.539597    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:54.356197 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:54.356160    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:54.466075 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:54.466045    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:54.560545 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:54.560511    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:54.738849 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:54.738761    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:36:55.668048 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:36:55.668015    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:00.233054 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:00.233019    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:00.333361 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:00.333321    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:00.875328 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:00.875286    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:01.033265 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:01.033233    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:37:01.154720 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:01.154646    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:01.450886 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:01.450801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:01.828136 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:01.828099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:02.367614 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:02.367579    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:02.654716 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:02.654635    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:02.987369 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:02.987276    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:03.088581 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:03.088550    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:03.380159 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:03.380122    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:03.797140 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:03.797060    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:04.003152 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:04.003119    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:04.765385 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:04.765348    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:04.852273 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:04.852233    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:04.960009 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:04.959977    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:05.820165 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:05.820123    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:06.503309 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:06.503271    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:07.375098 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:07.375062    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:07.995168 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:07.995134    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"]
Sep 17 03:37:07.995720 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:07.995698    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37ebb122-6f00-4efc-af79-259ed38ab82b" containerName="revoke-keys"
Sep 17 03:37:07.995772 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:07.995724    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="37ebb122-6f00-4efc-af79-259ed38ab82b" containerName="revoke-keys"
Sep 17 03:37:07.995772 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:07.995743    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="440f5e53-efda-4289-b0f4-32587e1d77aa" containerName="payload-processing"
Sep 17 03:37:07.995772 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:07.995753    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="440f5e53-efda-4289-b0f4-32587e1d77aa" containerName="payload-processing"
Sep 17 03:37:07.995868 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:07.995852    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="440f5e53-efda-4289-b0f4-32587e1d77aa" containerName="payload-processing"
Sep 17 03:37:07.995902 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:07.995870    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="37ebb122-6f00-4efc-af79-259ed38ab82b" containerName="revoke-keys"
Sep 17 03:37:08.003097 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.003073    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.005016 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.004985    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-57958f-serving-cert\""
Sep 17 03:37:08.009233 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.009207    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"]
Sep 17 03:37:08.025173 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.025133    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:08.078955 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.078921    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:08.115789 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.115757    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c75c2f50-4470-4207-b6d2-ad67461396a4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.115971 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.115798    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c75c2f50-4470-4207-b6d2-ad67461396a4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.115971 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.115819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c75c2f50-4470-4207-b6d2-ad67461396a4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.115971 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.115914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sf2w\" (UniqueName: \"kubernetes.io/projected/c75c2f50-4470-4207-b6d2-ad67461396a4-kube-api-access-2sf2w\") pod \"maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.180087 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.180054    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:08.217280 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.217246    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c75c2f50-4470-4207-b6d2-ad67461396a4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.217280 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.217284    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c75c2f50-4470-4207-b6d2-ad67461396a4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.217525 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.217323    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sf2w\" (UniqueName: \"kubernetes.io/projected/c75c2f50-4470-4207-b6d2-ad67461396a4-kube-api-access-2sf2w\") pod \"maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.217525 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.217384    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c75c2f50-4470-4207-b6d2-ad67461396a4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.217960 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.217932    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c75c2f50-4470-4207-b6d2-ad67461396a4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.219765 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.219744    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c75c2f50-4470-4207-b6d2-ad67461396a4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.219925 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.219903    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c75c2f50-4470-4207-b6d2-ad67461396a4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.224936 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.224912    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sf2w\" (UniqueName: \"kubernetes.io/projected/c75c2f50-4470-4207-b6d2-ad67461396a4-kube-api-access-2sf2w\") pod \"maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.316884 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.316844    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:08.473731 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.473695    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:08.571370 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.571292    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:08.602414 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.602385    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"]
Sep 17 03:37:08.603926 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:37:08.603898    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc75c2f50_4470_4207_b6d2_ad67461396a4.slice/crio-0d1bc3152a277aacd51e1f6d27a7b5473132de37ee9da30e4fff191f85934950 WatchSource:0}: Error finding container 0d1bc3152a277aacd51e1f6d27a7b5473132de37ee9da30e4fff191f85934950: Status 404 returned error can't find the container with id 0d1bc3152a277aacd51e1f6d27a7b5473132de37ee9da30e4fff191f85934950
Sep 17 03:37:08.787975 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:08.787941    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22" event={"ID":"c75c2f50-4470-4207-b6d2-ad67461396a4","Type":"ContainerStarted","Data":"0d1bc3152a277aacd51e1f6d27a7b5473132de37ee9da30e4fff191f85934950"}
Sep 17 03:37:09.802633 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:09.802594    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22" event={"ID":"c75c2f50-4470-4207-b6d2-ad67461396a4","Type":"ContainerStarted","Data":"4485f7ec3044dc75ca4972701c6f9a81cfd5e31ea3c85a732b9502aed455b2a9"}
Sep 17 03:37:09.803046 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:09.802690    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:09.821803 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:09.821755    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22" podStartSLOduration=2.48129374 podStartE2EDuration="2.821739871s" totalImagesPullingTime="340.446131ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:37:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:37:09.818534598 +0000 UTC m=+1058.163260677" watchObservedRunningTime="2026-09-17 03:37:09.821739871 +0000 UTC m=+1058.166465944"
Sep 17 03:37:09.846971 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:09.846927    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:13.262395 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:13.262354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:14.673870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:14.673837    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:15.814875 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:15.814848    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:37:18.265543 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.265505    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"]
Sep 17 03:37:18.268657 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.268638    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.270621 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.270594    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-214fea\"/\"kube-root-ca.crt\""
Sep 17 03:37:18.270756 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.270637    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-214fea\"/\"default-dockercfg-g7kkn\""
Sep 17 03:37:18.271075 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.271057    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-214fea\"/\"openshift-service-ca.crt\""
Sep 17 03:37:18.271161 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.271116    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-214fea\"/\"e2e-conflict-b-fed91830-kserve-self-signed-certs\""
Sep 17 03:37:18.276667 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.276362    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"]
Sep 17 03:37:18.412260 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.412227    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1a71d6c-6eb1-4657-a150-02ecdeacff77-tls-certs\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.412516 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.412277    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-tmp-dir\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.412516 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.412412    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-model-cache\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.412516 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.412504    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-dshm\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.412690 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.412537    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjsmn\" (UniqueName: \"kubernetes.io/projected/d1a71d6c-6eb1-4657-a150-02ecdeacff77-kube-api-access-qjsmn\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.412690 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.412570    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-home\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.513965 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.513931    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-model-cache\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.514150 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.513980    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-dshm\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.514150 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.514000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjsmn\" (UniqueName: \"kubernetes.io/projected/d1a71d6c-6eb1-4657-a150-02ecdeacff77-kube-api-access-qjsmn\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.514150 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.514029    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-home\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.514150 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.514074    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1a71d6c-6eb1-4657-a150-02ecdeacff77-tls-certs\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.514150 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.514118    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-tmp-dir\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.514530 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.514501    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-home\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.514530 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.514524    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-model-cache\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.514707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.514555    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-tmp-dir\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.516300 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.516245    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-dshm\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.516463 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.516424    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1a71d6c-6eb1-4657-a150-02ecdeacff77-tls-certs\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.521141 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.521120    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjsmn\" (UniqueName: \"kubernetes.io/projected/d1a71d6c-6eb1-4657-a150-02ecdeacff77-kube-api-access-qjsmn\") pod \"e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") " pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.565444 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.565400    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:18.579490 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.579450    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:18.853211 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:18.853186    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"]
Sep 17 03:37:18.855416 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:37:18.855389    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1a71d6c_6eb1_4657_a150_02ecdeacff77.slice/crio-ad2d3929de58f86b8fe842c3233fe831f87d911fd55b7f006e1e8cd0d7be9b59 WatchSource:0}: Error finding container ad2d3929de58f86b8fe842c3233fe831f87d911fd55b7f006e1e8cd0d7be9b59: Status 404 returned error can't find the container with id ad2d3929de58f86b8fe842c3233fe831f87d911fd55b7f006e1e8cd0d7be9b59
Sep 17 03:37:19.863845 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:19.863808    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5" event={"ID":"d1a71d6c-6eb1-4657-a150-02ecdeacff77","Type":"ContainerStarted","Data":"d3287c1e78420dbcecf2e20abb2148dc93e423d05c4748d8411a2fa5cbb81aa5"}
Sep 17 03:37:19.863845 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:19.863854    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5" event={"ID":"d1a71d6c-6eb1-4657-a150-02ecdeacff77","Type":"ContainerStarted","Data":"ad2d3929de58f86b8fe842c3233fe831f87d911fd55b7f006e1e8cd0d7be9b59"}
Sep 17 03:37:19.878809 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:19.878749    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5" podStartSLOduration=1.8787259760000001 podStartE2EDuration="1.878725976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:37:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:37:19.87753907 +0000 UTC m=+1068.222265149" watchObservedRunningTime="2026-09-17 03:37:19.878725976 +0000 UTC m=+1068.223452055"
Sep 17 03:37:23.658120 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:23.658088    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"]
Sep 17 03:37:23.676143 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:23.676112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:27.343130 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:27.343089    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:37:28.580368 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:28.580332    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:28.580846 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:28.580394    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:28.594728 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:28.594689    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:28.878630 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:28.878533    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:28.929215 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:28.929185    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:37:30.346307 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:30.346272    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:31.236327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:31.236289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:31.992124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:31.992087    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:32.906403 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:32.906356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:33.835450 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:33.835385    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:34.608057 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:34.608020    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:34.735038 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:34.734986    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:35.126638 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:35.126601    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:35.458503 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:35.458388    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:35.881870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:35.881827    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:38.541126 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:38.541092    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:38.678033 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:38.677989    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g" podUID="4a14a289-96db-4bfd-943a-e3863008ffda" containerName="main" containerID="cri-o://b0b6d699cbfa131c3063f48a820656692a598d00a24d7daf29cdde632c703cae" gracePeriod=45
Sep 17 03:37:38.780174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:38.780098    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:38.875611 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:38.875572    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:38.940568 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:38.940543    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:37:38.972332 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:38.972304    2576 generic.go:356] "Generic (PLEG): container finished" podID="4a14a289-96db-4bfd-943a-e3863008ffda" containerID="b0b6d699cbfa131c3063f48a820656692a598d00a24d7daf29cdde632c703cae" exitCode=0
Sep 17 03:37:38.972523 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:38.972364    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"
Sep 17 03:37:38.972523 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:38.972391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g" event={"ID":"4a14a289-96db-4bfd-943a-e3863008ffda","Type":"ContainerDied","Data":"b0b6d699cbfa131c3063f48a820656692a598d00a24d7daf29cdde632c703cae"}
Sep 17 03:37:38.972523 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:38.972449    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g" event={"ID":"4a14a289-96db-4bfd-943a-e3863008ffda","Type":"ContainerDied","Data":"12c1ba6410455ab5ac41325f1529cf317d2db51fe2ebc1ad1b77e9b4d42f0324"}
Sep 17 03:37:38.972523 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:38.972473    2576 scope.go:117] "RemoveContainer" containerID="b0b6d699cbfa131c3063f48a820656692a598d00a24d7daf29cdde632c703cae"
Sep 17 03:37:38.982299 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:38.982277    2576 scope.go:117] "RemoveContainer" containerID="b0b6d699cbfa131c3063f48a820656692a598d00a24d7daf29cdde632c703cae"
Sep 17 03:37:38.982663 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:37:38.982639    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0b6d699cbfa131c3063f48a820656692a598d00a24d7daf29cdde632c703cae\": container with ID starting with b0b6d699cbfa131c3063f48a820656692a598d00a24d7daf29cdde632c703cae not found: ID does not exist" containerID="b0b6d699cbfa131c3063f48a820656692a598d00a24d7daf29cdde632c703cae"
Sep 17 03:37:38.982764 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:38.982679    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0b6d699cbfa131c3063f48a820656692a598d00a24d7daf29cdde632c703cae"} err="failed to get container status \"b0b6d699cbfa131c3063f48a820656692a598d00a24d7daf29cdde632c703cae\": rpc error: code = NotFound desc = could not find container \"b0b6d699cbfa131c3063f48a820656692a598d00a24d7daf29cdde632c703cae\": container with ID starting with b0b6d699cbfa131c3063f48a820656692a598d00a24d7daf29cdde632c703cae not found: ID does not exist"
Sep 17 03:37:39.006082 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.006054    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a14a289-96db-4bfd-943a-e3863008ffda-tls-certs\") pod \"4a14a289-96db-4bfd-943a-e3863008ffda\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") "
Sep 17 03:37:39.006267 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.006099    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-home\") pod \"4a14a289-96db-4bfd-943a-e3863008ffda\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") "
Sep 17 03:37:39.006267 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.006124    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-dshm\") pod \"4a14a289-96db-4bfd-943a-e3863008ffda\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") "
Sep 17 03:37:39.006267 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.006147    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-tmp-dir\") pod \"4a14a289-96db-4bfd-943a-e3863008ffda\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") "
Sep 17 03:37:39.006267 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.006212    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-model-cache\") pod \"4a14a289-96db-4bfd-943a-e3863008ffda\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") "
Sep 17 03:37:39.006503 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.006272    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t9mx\" (UniqueName: \"kubernetes.io/projected/4a14a289-96db-4bfd-943a-e3863008ffda-kube-api-access-8t9mx\") pod \"4a14a289-96db-4bfd-943a-e3863008ffda\" (UID: \"4a14a289-96db-4bfd-943a-e3863008ffda\") "
Sep 17 03:37:39.006503 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.006388    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-home" (OuterVolumeSpecName: "home") pod "4a14a289-96db-4bfd-943a-e3863008ffda" (UID: "4a14a289-96db-4bfd-943a-e3863008ffda"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:37:39.006503 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.006461    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4a14a289-96db-4bfd-943a-e3863008ffda" (UID: "4a14a289-96db-4bfd-943a-e3863008ffda"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:37:39.006711 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.006540    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-model-cache" (OuterVolumeSpecName: "model-cache") pod "4a14a289-96db-4bfd-943a-e3863008ffda" (UID: "4a14a289-96db-4bfd-943a-e3863008ffda"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:37:39.006711 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.006630    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-home\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:37:39.006711 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.006676    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-tmp-dir\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:37:39.006711 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.006691    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-model-cache\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:37:39.008520 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.008494    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a14a289-96db-4bfd-943a-e3863008ffda-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4a14a289-96db-4bfd-943a-e3863008ffda" (UID: "4a14a289-96db-4bfd-943a-e3863008ffda"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:37:39.008738 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.008715    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-dshm" (OuterVolumeSpecName: "dshm") pod "4a14a289-96db-4bfd-943a-e3863008ffda" (UID: "4a14a289-96db-4bfd-943a-e3863008ffda"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:37:39.008820 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.008792    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a14a289-96db-4bfd-943a-e3863008ffda-kube-api-access-8t9mx" (OuterVolumeSpecName: "kube-api-access-8t9mx") pod "4a14a289-96db-4bfd-943a-e3863008ffda" (UID: "4a14a289-96db-4bfd-943a-e3863008ffda"). InnerVolumeSpecName "kube-api-access-8t9mx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:37:39.107705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.107668    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t9mx\" (UniqueName: \"kubernetes.io/projected/4a14a289-96db-4bfd-943a-e3863008ffda-kube-api-access-8t9mx\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:37:39.107705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.107696    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a14a289-96db-4bfd-943a-e3863008ffda-tls-certs\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:37:39.107705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.107705    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a14a289-96db-4bfd-943a-e3863008ffda-dshm\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:37:39.293581 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.293549    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"]
Sep 17 03:37:39.300136 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.300109    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-297324/test-model-297324-kserve-7fd46548c-rdj4g"]
Sep 17 03:37:39.480811 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:39.480586    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:40.293915 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:40.293879    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a14a289-96db-4bfd-943a-e3863008ffda" path="/var/lib/kubelet/pods/4a14a289-96db-4bfd-943a-e3863008ffda/volumes"
Sep 17 03:37:40.398082 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:40.398036    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:41.126194 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:41.126151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:41.828399 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:41.828365    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:42.615785 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.615749    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"]
Sep 17 03:37:42.616182 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.616165    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a14a289-96db-4bfd-943a-e3863008ffda" containerName="main"
Sep 17 03:37:42.616182 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.616181    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a14a289-96db-4bfd-943a-e3863008ffda" containerName="main"
Sep 17 03:37:42.616335 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.616289    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a14a289-96db-4bfd-943a-e3863008ffda" containerName="main"
Sep 17 03:37:42.620758 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.620732    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:42.622583 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.622559    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-fc820e-serving-cert\""
Sep 17 03:37:42.631172 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.631143    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"]
Sep 17 03:37:42.640129 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.640100    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6lr5\" (UniqueName: \"kubernetes.io/projected/d8f4c72c-cf3d-4e79-98a5-04948958dc60-kube-api-access-z6lr5\") pod \"maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:42.640249 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.640145    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:42.640329 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.640310    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f4c72c-cf3d-4e79-98a5-04948958dc60-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:42.640375 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.640338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:42.642550 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.642532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:42.745377 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.741040    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f4c72c-cf3d-4e79-98a5-04948958dc60-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:42.745377 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.741085    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:42.745377 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.741143    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6lr5\" (UniqueName: \"kubernetes.io/projected/d8f4c72c-cf3d-4e79-98a5-04948958dc60-kube-api-access-z6lr5\") pod \"maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:42.745377 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.741168    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:42.745377 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:37:42.741315    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-serving-cert: secret "maas-api-e2e-shared-b-w2-fc820e-serving-cert" not found
Sep 17 03:37:42.745377 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:37:42.741408    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-maas-api-tls podName:d8f4c72c-cf3d-4e79-98a5-04948958dc60 nodeName:}" failed. No retries permitted until 2026-09-17 03:37:43.241386468 +0000 UTC m=+1091.586112549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-maas-api-tls") pod "maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4" (UID: "d8f4c72c-cf3d-4e79-98a5-04948958dc60") : secret "maas-api-e2e-shared-b-w2-fc820e-serving-cert" not found
Sep 17 03:37:42.745377 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.742390    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f4c72c-cf3d-4e79-98a5-04948958dc60-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:42.748468 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.746970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:42.749094 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.749027    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:42.752086 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:42.752064    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6lr5\" (UniqueName: \"kubernetes.io/projected/d8f4c72c-cf3d-4e79-98a5-04948958dc60-kube-api-access-z6lr5\") pod \"maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:43.016213 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.016129    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:43.226131 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.226096    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:43.245492 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.245451    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:43.248102 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.248080    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:43.325019 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.324984    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:43.331241 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.331213    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"]
Sep 17 03:37:43.334971 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.334954    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"
Sep 17 03:37:43.337234 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.337206    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-fc820e\""
Sep 17 03:37:43.337336 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.337264    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-fc820e-dockercfg-7hbr8\""
Sep 17 03:37:43.345282 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.345254    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"]
Sep 17 03:37:43.364917 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.364889    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"]
Sep 17 03:37:43.368273 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.368256    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"
Sep 17 03:37:43.378374 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.378345    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"]
Sep 17 03:37:43.447905 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.447871    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krqrk\" (UniqueName: \"kubernetes.io/projected/cb91a032-966e-4f3e-81ad-aeb6decaf99d-kube-api-access-krqrk\") pod \"payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5\" (UID: \"cb91a032-966e-4f3e-81ad-aeb6decaf99d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"
Sep 17 03:37:43.448164 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.447929    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb91a032-966e-4f3e-81ad-aeb6decaf99d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5\" (UID: \"cb91a032-966e-4f3e-81ad-aeb6decaf99d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"
Sep 17 03:37:43.448164 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.447961    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b01d3c12-8dc6-42da-a6ad-df793133628e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n\" (UID: \"b01d3c12-8dc6-42da-a6ad-df793133628e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"
Sep 17 03:37:43.448164 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.447989    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78lkx\" (UniqueName: \"kubernetes.io/projected/b01d3c12-8dc6-42da-a6ad-df793133628e-kube-api-access-78lkx\") pod \"payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n\" (UID: \"b01d3c12-8dc6-42da-a6ad-df793133628e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"
Sep 17 03:37:43.533959 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.533929    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:43.549184 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.549143    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb91a032-966e-4f3e-81ad-aeb6decaf99d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5\" (UID: \"cb91a032-966e-4f3e-81ad-aeb6decaf99d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"
Sep 17 03:37:43.549325 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.549188    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b01d3c12-8dc6-42da-a6ad-df793133628e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n\" (UID: \"b01d3c12-8dc6-42da-a6ad-df793133628e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"
Sep 17 03:37:43.549325 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.549214    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78lkx\" (UniqueName: \"kubernetes.io/projected/b01d3c12-8dc6-42da-a6ad-df793133628e-kube-api-access-78lkx\") pod \"payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n\" (UID: \"b01d3c12-8dc6-42da-a6ad-df793133628e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"
Sep 17 03:37:43.549497 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.549349    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krqrk\" (UniqueName: \"kubernetes.io/projected/cb91a032-966e-4f3e-81ad-aeb6decaf99d-kube-api-access-krqrk\") pod \"payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5\" (UID: \"cb91a032-966e-4f3e-81ad-aeb6decaf99d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"
Sep 17 03:37:43.550029 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.550007    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb91a032-966e-4f3e-81ad-aeb6decaf99d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5\" (UID: \"cb91a032-966e-4f3e-81ad-aeb6decaf99d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"
Sep 17 03:37:43.550471 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.550419    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b01d3c12-8dc6-42da-a6ad-df793133628e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n\" (UID: \"b01d3c12-8dc6-42da-a6ad-df793133628e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"
Sep 17 03:37:43.558380 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.558357    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krqrk\" (UniqueName: \"kubernetes.io/projected/cb91a032-966e-4f3e-81ad-aeb6decaf99d-kube-api-access-krqrk\") pod \"payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5\" (UID: \"cb91a032-966e-4f3e-81ad-aeb6decaf99d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"
Sep 17 03:37:43.558594 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.558575    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78lkx\" (UniqueName: \"kubernetes.io/projected/b01d3c12-8dc6-42da-a6ad-df793133628e-kube-api-access-78lkx\") pod \"payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n\" (UID: \"b01d3c12-8dc6-42da-a6ad-df793133628e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"
Sep 17 03:37:43.647183 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.647151    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"
Sep 17 03:37:43.680535 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.680500    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"
Sep 17 03:37:43.810029 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.809995    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"]
Sep 17 03:37:43.811240 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:37:43.811219    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8f4c72c_cf3d_4e79_98a5_04948958dc60.slice/crio-2d7946368067051b4d5421d04eae4f4ed3ebb07d0216ae34bb8054a3344b8e78 WatchSource:0}: Error finding container 2d7946368067051b4d5421d04eae4f4ed3ebb07d0216ae34bb8054a3344b8e78: Status 404 returned error can't find the container with id 2d7946368067051b4d5421d04eae4f4ed3ebb07d0216ae34bb8054a3344b8e78
Sep 17 03:37:43.920594 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.920563    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"]
Sep 17 03:37:43.921121 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:37:43.921092    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb91a032_966e_4f3e_81ad_aeb6decaf99d.slice/crio-ef7211124b57d226bc97bc74b76f0ac5c8bb03855f9f1e2a48d15a252342a3d9 WatchSource:0}: Error finding container ef7211124b57d226bc97bc74b76f0ac5c8bb03855f9f1e2a48d15a252342a3d9: Status 404 returned error can't find the container with id ef7211124b57d226bc97bc74b76f0ac5c8bb03855f9f1e2a48d15a252342a3d9
Sep 17 03:37:43.963957 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:43.963733    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"]
Sep 17 03:37:43.967299 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:37:43.967085    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb01d3c12_8dc6_42da_a6ad_df793133628e.slice/crio-a90db0c3fd5c4c12467a4059bd3a6e15a2d7697e9fe46fdf01d5ef8ecffe038e WatchSource:0}: Error finding container a90db0c3fd5c4c12467a4059bd3a6e15a2d7697e9fe46fdf01d5ef8ecffe038e: Status 404 returned error can't find the container with id a90db0c3fd5c4c12467a4059bd3a6e15a2d7697e9fe46fdf01d5ef8ecffe038e
Sep 17 03:37:44.003471 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:44.003425    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n" event={"ID":"b01d3c12-8dc6-42da-a6ad-df793133628e","Type":"ContainerStarted","Data":"a90db0c3fd5c4c12467a4059bd3a6e15a2d7697e9fe46fdf01d5ef8ecffe038e"}
Sep 17 03:37:44.005226 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:44.005198    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5" event={"ID":"cb91a032-966e-4f3e-81ad-aeb6decaf99d","Type":"ContainerStarted","Data":"ef7211124b57d226bc97bc74b76f0ac5c8bb03855f9f1e2a48d15a252342a3d9"}
Sep 17 03:37:44.006488 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:44.006467    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4" event={"ID":"d8f4c72c-cf3d-4e79-98a5-04948958dc60","Type":"ContainerStarted","Data":"2d7946368067051b4d5421d04eae4f4ed3ebb07d0216ae34bb8054a3344b8e78"}
Sep 17 03:37:44.585095 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:44.585056    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:45.015262 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:45.015152    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n" event={"ID":"b01d3c12-8dc6-42da-a6ad-df793133628e","Type":"ContainerStarted","Data":"1d69b06f3bf0894b7ce16b3c7e393d7bebef72ff18386a0e77c5c587810e6029"}
Sep 17 03:37:45.015421 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:45.015349    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"
Sep 17 03:37:45.016813 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:45.016791    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5" event={"ID":"cb91a032-966e-4f3e-81ad-aeb6decaf99d","Type":"ContainerStarted","Data":"17be8b73dbe221d120717e9e268516c22d043f80f1cab93aa2f1d9f49b4b9a1f"}
Sep 17 03:37:45.016953 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:45.016841    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"
Sep 17 03:37:45.018508 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:45.018487    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4" event={"ID":"d8f4c72c-cf3d-4e79-98a5-04948958dc60","Type":"ContainerStarted","Data":"c2c9d33ece956490688133f5c5c4b3286ce7921056acbd2fe559c752157f593f"}
Sep 17 03:37:45.018625 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:45.018609    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:45.031982 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:45.031936    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n" podStartSLOduration=2.031919985 podStartE2EDuration="2.031919985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:37:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:37:45.029180539 +0000 UTC m=+1093.373906620" watchObservedRunningTime="2026-09-17 03:37:45.031919985 +0000 UTC m=+1093.376646118"
Sep 17 03:37:45.044904 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:45.044854    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4" podStartSLOduration=2.569176675 podStartE2EDuration="3.044834669s" totalImagesPullingTime="475.657994ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:37:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:37:45.043177989 +0000 UTC m=+1093.387904069" watchObservedRunningTime="2026-09-17 03:37:45.044834669 +0000 UTC m=+1093.389560747"
Sep 17 03:37:45.058626 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:45.058563    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5" podStartSLOduration=2.05854185 podStartE2EDuration="2.05854185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:37:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:37:45.056839441 +0000 UTC m=+1093.401565520" watchObservedRunningTime="2026-09-17 03:37:45.05854185 +0000 UTC m=+1093.403267929"
Sep 17 03:37:51.030494 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:51.030463    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:37:54.892893 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:54.892861    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:55.600675 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:55.600626    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:56.026883 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:56.026795    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"
Sep 17 03:37:56.027326 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:56.026967    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"
Sep 17 03:37:56.446982 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:56.446947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:57.209086 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:57.209051    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:57.841622 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:57.841571    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:58.678532 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:58.678501    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:37:59.382713 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:37:59.382679    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:00.011566 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:00.011530    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:00.633839 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:00.633796    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:01.333816 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:01.333780    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:02.003961 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:02.003926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:02.437935 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:02.437905    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:38:02.734357 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:02.734273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:03.384148 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:03.384111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:03.713941 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:03.713848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:04.196607 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:04.196571    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:04.735164 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:04.735131    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"]
Sep 17 03:38:04.735642 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:04.735373    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d" podUID="f6fcd273-1108-4a23-856a-4e328f7a7834" containerName="maas-api" containerID="cri-o://5aa1723dccf6dbb58e731b0355e855d844c53823af8c38e8a4f86d406d547c12" gracePeriod=30
Sep 17 03:38:04.752996 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:04.752972    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:04.853629 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:04.853599    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:04.935607 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:04.935573    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:04.987415 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:04.987353    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:38:04.991551 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:04.991526    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:05.135425 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.135389    2576 generic.go:356] "Generic (PLEG): container finished" podID="f6fcd273-1108-4a23-856a-4e328f7a7834" containerID="5aa1723dccf6dbb58e731b0355e855d844c53823af8c38e8a4f86d406d547c12" exitCode=0
Sep 17 03:38:05.135604 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.135447    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"
Sep 17 03:38:05.135604 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.135473    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d" event={"ID":"f6fcd273-1108-4a23-856a-4e328f7a7834","Type":"ContainerDied","Data":"5aa1723dccf6dbb58e731b0355e855d844c53823af8c38e8a4f86d406d547c12"}
Sep 17 03:38:05.135604 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.135533    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d" event={"ID":"f6fcd273-1108-4a23-856a-4e328f7a7834","Type":"ContainerDied","Data":"7c4c2f9cc727a9895650ba4b3cce3dc37648a79b6ec003f0860180cd59947ecd"}
Sep 17 03:38:05.135604 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.135564    2576 scope.go:117] "RemoveContainer" containerID="5aa1723dccf6dbb58e731b0355e855d844c53823af8c38e8a4f86d406d547c12"
Sep 17 03:38:05.145832 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.145814    2576 scope.go:117] "RemoveContainer" containerID="5aa1723dccf6dbb58e731b0355e855d844c53823af8c38e8a4f86d406d547c12"
Sep 17 03:38:05.146149 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:38:05.146128    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5aa1723dccf6dbb58e731b0355e855d844c53823af8c38e8a4f86d406d547c12\": container with ID starting with 5aa1723dccf6dbb58e731b0355e855d844c53823af8c38e8a4f86d406d547c12 not found: ID does not exist" containerID="5aa1723dccf6dbb58e731b0355e855d844c53823af8c38e8a4f86d406d547c12"
Sep 17 03:38:05.146208 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.146160    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5aa1723dccf6dbb58e731b0355e855d844c53823af8c38e8a4f86d406d547c12"} err="failed to get container status \"5aa1723dccf6dbb58e731b0355e855d844c53823af8c38e8a4f86d406d547c12\": rpc error: code = NotFound desc = could not find container \"5aa1723dccf6dbb58e731b0355e855d844c53823af8c38e8a4f86d406d547c12\": container with ID starting with 5aa1723dccf6dbb58e731b0355e855d844c53823af8c38e8a4f86d406d547c12 not found: ID does not exist"
Sep 17 03:38:05.160461 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.160416    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5bgd\" (UniqueName: \"kubernetes.io/projected/f6fcd273-1108-4a23-856a-4e328f7a7834-kube-api-access-w5bgd\") pod \"f6fcd273-1108-4a23-856a-4e328f7a7834\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") "
Sep 17 03:38:05.160613 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.160493    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6fcd273-1108-4a23-856a-4e328f7a7834-openshift-service-ca\") pod \"f6fcd273-1108-4a23-856a-4e328f7a7834\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") "
Sep 17 03:38:05.160613 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.160548    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-maas-api-tls\") pod \"f6fcd273-1108-4a23-856a-4e328f7a7834\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") "
Sep 17 03:38:05.160722 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.160668    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-metrics-certs\") pod \"f6fcd273-1108-4a23-856a-4e328f7a7834\" (UID: \"f6fcd273-1108-4a23-856a-4e328f7a7834\") "
Sep 17 03:38:05.160863 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.160837    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6fcd273-1108-4a23-856a-4e328f7a7834-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f6fcd273-1108-4a23-856a-4e328f7a7834" (UID: "f6fcd273-1108-4a23-856a-4e328f7a7834"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:38:05.160934 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.160916    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6fcd273-1108-4a23-856a-4e328f7a7834-openshift-service-ca\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:38:05.162809 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.162777    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f6fcd273-1108-4a23-856a-4e328f7a7834" (UID: "f6fcd273-1108-4a23-856a-4e328f7a7834"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:38:05.162809 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.162798    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f6fcd273-1108-4a23-856a-4e328f7a7834" (UID: "f6fcd273-1108-4a23-856a-4e328f7a7834"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:38:05.163024 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.162884    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6fcd273-1108-4a23-856a-4e328f7a7834-kube-api-access-w5bgd" (OuterVolumeSpecName: "kube-api-access-w5bgd") pod "f6fcd273-1108-4a23-856a-4e328f7a7834" (UID: "f6fcd273-1108-4a23-856a-4e328f7a7834"). InnerVolumeSpecName "kube-api-access-w5bgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:38:05.261573 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.261487    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-metrics-certs\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:38:05.261573 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.261518    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5bgd\" (UniqueName: \"kubernetes.io/projected/f6fcd273-1108-4a23-856a-4e328f7a7834-kube-api-access-w5bgd\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:38:05.261573 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.261527    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6fcd273-1108-4a23-856a-4e328f7a7834-maas-api-tls\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:38:05.275734 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.275706    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:05.398987 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.398950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:05.476028 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.475994    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"]
Sep 17 03:38:05.484925 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:05.484889    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-297324-64b8bf9c45-j5f4d"]
Sep 17 03:38:06.294583 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:06.294549    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6fcd273-1108-4a23-856a-4e328f7a7834" path="/var/lib/kubelet/pods/f6fcd273-1108-4a23-856a-4e328f7a7834/volumes"
Sep 17 03:38:06.311193 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:06.311163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:07.015049 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:07.015015    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:07.840357 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:07.840322    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:08.542060 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:08.542029    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:09.238037 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:09.237987    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:10.953139 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:10.953107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:11.010511 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:11.010479    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:11.204331 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:11.204245    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:12.242419 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:12.242386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:12.971533 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:12.971491    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:13.072474 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:13.072410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:13.247261 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:13.247184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:13.392878 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:13.392847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:13.884679 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:13.884643    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:14.521078 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:14.521048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:15.517212 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:15.517183    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:16.325156 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:16.325122    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:16.747491 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:16.747406    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:16.847805 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:16.847772    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:17.127043 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:17.127008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:18.041861 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:18.041827    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:18.814050 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:18.814015    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:19.511279 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:19.511244    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:20.441765 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:20.441732    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:21.234822 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.234788    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:21.290212 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.290172    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:21.693045 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.693008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:21.777063 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.777017    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:21.828456 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.828389    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"]
Sep 17 03:38:21.829054 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.829032    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6fcd273-1108-4a23-856a-4e328f7a7834" containerName="maas-api"
Sep 17 03:38:21.829104 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.829061    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6fcd273-1108-4a23-856a-4e328f7a7834" containerName="maas-api"
Sep 17 03:38:21.829201 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.829189    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6fcd273-1108-4a23-856a-4e328f7a7834" containerName="maas-api"
Sep 17 03:38:21.834309 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.834282    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"
Sep 17 03:38:21.836418 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.836393    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c10ab8fb\""
Sep 17 03:38:21.837076 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.836839    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c10ab8fb-dockercfg-cb7g2\""
Sep 17 03:38:21.839761 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.839712    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"]
Sep 17 03:38:21.910720 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.910686    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w65dw\" (UniqueName: \"kubernetes.io/projected/1f9dd19b-fbaa-4266-b690-e45e3b636372-kube-api-access-w65dw\") pod \"payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j\" (UID: \"1f9dd19b-fbaa-4266-b690-e45e3b636372\") " pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"
Sep 17 03:38:21.910910 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.910801    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f9dd19b-fbaa-4266-b690-e45e3b636372-plugins-config-volume\") pod \"payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j\" (UID: \"1f9dd19b-fbaa-4266-b690-e45e3b636372\") " pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"
Sep 17 03:38:21.928236 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.928204    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:21.960349 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.960256    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"]
Sep 17 03:38:21.960718 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:21.960535    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg" podUID="674d2736-fcb8-420e-b66a-dfd58993dcbf" containerName="maas-api" containerID="cri-o://9b9295f16dc002ccf18351349a21794051924aefbdb21b2012279273e827bdea" gracePeriod=30
Sep 17 03:38:22.009343 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.009310    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"]
Sep 17 03:38:22.009596 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.009566    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr" podUID="813bbf20-e4e8-4476-a390-e374a0e3c2db" containerName="payload-processing" containerID="cri-o://0d2540a5019665f5e91ba8dcf98f959d88eed853e3faa7888ba86fcaed046d78" gracePeriod=30
Sep 17 03:38:22.012407 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.011682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w65dw\" (UniqueName: \"kubernetes.io/projected/1f9dd19b-fbaa-4266-b690-e45e3b636372-kube-api-access-w65dw\") pod \"payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j\" (UID: \"1f9dd19b-fbaa-4266-b690-e45e3b636372\") " pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"
Sep 17 03:38:22.012407 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.011730    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f9dd19b-fbaa-4266-b690-e45e3b636372-plugins-config-volume\") pod \"payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j\" (UID: \"1f9dd19b-fbaa-4266-b690-e45e3b636372\") " pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"
Sep 17 03:38:22.013081 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.012788    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f9dd19b-fbaa-4266-b690-e45e3b636372-plugins-config-volume\") pod \"payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j\" (UID: \"1f9dd19b-fbaa-4266-b690-e45e3b636372\") " pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"
Sep 17 03:38:22.014910 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.014549    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:22.017413 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.017394    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"]
Sep 17 03:38:22.017902 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.017874    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62" podUID="f8a47a4e-01e9-4fde-973d-8106d545b716" containerName="payload-pre-processing" containerID="cri-o://827b7f67f8f39c0c250739ef609a14a9ad1e60cfba9e3a55b8d501a8037940c6" gracePeriod=30
Sep 17 03:38:22.023510 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.023488    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w65dw\" (UniqueName: \"kubernetes.io/projected/1f9dd19b-fbaa-4266-b690-e45e3b636372-kube-api-access-w65dw\") pod \"payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j\" (UID: \"1f9dd19b-fbaa-4266-b690-e45e3b636372\") " pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"
Sep 17 03:38:22.073496 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.073449    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:22.130343 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.130300    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg" podUID="674d2736-fcb8-420e-b66a-dfd58993dcbf" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.42:8443/health\": dial tcp 10.134.0.42:8443: connect: connection refused"
Sep 17 03:38:22.147360 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.147326    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"
Sep 17 03:38:22.200879 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.200684    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:22.246260 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.246151    2576 generic.go:356] "Generic (PLEG): container finished" podID="813bbf20-e4e8-4476-a390-e374a0e3c2db" containerID="0d2540a5019665f5e91ba8dcf98f959d88eed853e3faa7888ba86fcaed046d78" exitCode=0
Sep 17 03:38:22.246817 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.246266    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr" event={"ID":"813bbf20-e4e8-4476-a390-e374a0e3c2db","Type":"ContainerDied","Data":"0d2540a5019665f5e91ba8dcf98f959d88eed853e3faa7888ba86fcaed046d78"}
Sep 17 03:38:22.249333 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.249160    2576 generic.go:356] "Generic (PLEG): container finished" podID="f8a47a4e-01e9-4fde-973d-8106d545b716" containerID="827b7f67f8f39c0c250739ef609a14a9ad1e60cfba9e3a55b8d501a8037940c6" exitCode=0
Sep 17 03:38:22.249333 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.249274    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62" event={"ID":"f8a47a4e-01e9-4fde-973d-8106d545b716","Type":"ContainerDied","Data":"827b7f67f8f39c0c250739ef609a14a9ad1e60cfba9e3a55b8d501a8037940c6"}
Sep 17 03:38:22.251820 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.251659    2576 generic.go:356] "Generic (PLEG): container finished" podID="674d2736-fcb8-420e-b66a-dfd58993dcbf" containerID="9b9295f16dc002ccf18351349a21794051924aefbdb21b2012279273e827bdea" exitCode=0
Sep 17 03:38:22.251820 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.251758    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg" event={"ID":"674d2736-fcb8-420e-b66a-dfd58993dcbf","Type":"ContainerDied","Data":"9b9295f16dc002ccf18351349a21794051924aefbdb21b2012279273e827bdea"}
Sep 17 03:38:22.290002 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.289717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:22.316282 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.316100    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:38:22.356460 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.356418    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"
Sep 17 03:38:22.365110 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.365086    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"
Sep 17 03:38:22.416511 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.416477    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv6cq\" (UniqueName: \"kubernetes.io/projected/f8a47a4e-01e9-4fde-973d-8106d545b716-kube-api-access-hv6cq\") pod \"f8a47a4e-01e9-4fde-973d-8106d545b716\" (UID: \"f8a47a4e-01e9-4fde-973d-8106d545b716\") "
Sep 17 03:38:22.416688 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.416525    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/674d2736-fcb8-420e-b66a-dfd58993dcbf-metrics-certs\") pod \"674d2736-fcb8-420e-b66a-dfd58993dcbf\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") "
Sep 17 03:38:22.416688 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.416592    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/674d2736-fcb8-420e-b66a-dfd58993dcbf-maas-api-tls\") pod \"674d2736-fcb8-420e-b66a-dfd58993dcbf\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") "
Sep 17 03:38:22.416688 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.416611    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkv2j\" (UniqueName: \"kubernetes.io/projected/813bbf20-e4e8-4476-a390-e374a0e3c2db-kube-api-access-xkv2j\") pod \"813bbf20-e4e8-4476-a390-e374a0e3c2db\" (UID: \"813bbf20-e4e8-4476-a390-e374a0e3c2db\") "
Sep 17 03:38:22.416688 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.416643    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/674d2736-fcb8-420e-b66a-dfd58993dcbf-openshift-service-ca\") pod \"674d2736-fcb8-420e-b66a-dfd58993dcbf\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") "
Sep 17 03:38:22.416688 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.416666    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c45rq\" (UniqueName: \"kubernetes.io/projected/674d2736-fcb8-420e-b66a-dfd58993dcbf-kube-api-access-c45rq\") pod \"674d2736-fcb8-420e-b66a-dfd58993dcbf\" (UID: \"674d2736-fcb8-420e-b66a-dfd58993dcbf\") "
Sep 17 03:38:22.416985 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.416789    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8a47a4e-01e9-4fde-973d-8106d545b716-plugins-config-volume\") pod \"f8a47a4e-01e9-4fde-973d-8106d545b716\" (UID: \"f8a47a4e-01e9-4fde-973d-8106d545b716\") "
Sep 17 03:38:22.416985 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.416855    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/813bbf20-e4e8-4476-a390-e374a0e3c2db-plugins-config-volume\") pod \"813bbf20-e4e8-4476-a390-e374a0e3c2db\" (UID: \"813bbf20-e4e8-4476-a390-e374a0e3c2db\") "
Sep 17 03:38:22.417083 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.417037    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/674d2736-fcb8-420e-b66a-dfd58993dcbf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "674d2736-fcb8-420e-b66a-dfd58993dcbf" (UID: "674d2736-fcb8-420e-b66a-dfd58993dcbf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:38:22.417311 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.417267    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/674d2736-fcb8-420e-b66a-dfd58993dcbf-openshift-service-ca\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:38:22.417473 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.417340    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8a47a4e-01e9-4fde-973d-8106d545b716-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f8a47a4e-01e9-4fde-973d-8106d545b716" (UID: "f8a47a4e-01e9-4fde-973d-8106d545b716"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:38:22.417473 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.417387    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/813bbf20-e4e8-4476-a390-e374a0e3c2db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "813bbf20-e4e8-4476-a390-e374a0e3c2db" (UID: "813bbf20-e4e8-4476-a390-e374a0e3c2db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:38:22.418923 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.418896    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/813bbf20-e4e8-4476-a390-e374a0e3c2db-kube-api-access-xkv2j" (OuterVolumeSpecName: "kube-api-access-xkv2j") pod "813bbf20-e4e8-4476-a390-e374a0e3c2db" (UID: "813bbf20-e4e8-4476-a390-e374a0e3c2db"). InnerVolumeSpecName "kube-api-access-xkv2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:38:22.419025 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.418900    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/674d2736-fcb8-420e-b66a-dfd58993dcbf-kube-api-access-c45rq" (OuterVolumeSpecName: "kube-api-access-c45rq") pod "674d2736-fcb8-420e-b66a-dfd58993dcbf" (UID: "674d2736-fcb8-420e-b66a-dfd58993dcbf"). InnerVolumeSpecName "kube-api-access-c45rq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:38:22.419025 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.418907    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/674d2736-fcb8-420e-b66a-dfd58993dcbf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "674d2736-fcb8-420e-b66a-dfd58993dcbf" (UID: "674d2736-fcb8-420e-b66a-dfd58993dcbf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:38:22.419150 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.419129    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a47a4e-01e9-4fde-973d-8106d545b716-kube-api-access-hv6cq" (OuterVolumeSpecName: "kube-api-access-hv6cq") pod "f8a47a4e-01e9-4fde-973d-8106d545b716" (UID: "f8a47a4e-01e9-4fde-973d-8106d545b716"). InnerVolumeSpecName "kube-api-access-hv6cq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:38:22.419200 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.419131    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/674d2736-fcb8-420e-b66a-dfd58993dcbf-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "674d2736-fcb8-420e-b66a-dfd58993dcbf" (UID: "674d2736-fcb8-420e-b66a-dfd58993dcbf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:38:22.481474 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.481441    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"]
Sep 17 03:38:22.483036 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:38:22.483007    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9dd19b_fbaa_4266_b690_e45e3b636372.slice/crio-807693f9c0ee58e46254639d73644d377e504577b3a676a82ae1631ec2340927 WatchSource:0}: Error finding container 807693f9c0ee58e46254639d73644d377e504577b3a676a82ae1631ec2340927: Status 404 returned error can't find the container with id 807693f9c0ee58e46254639d73644d377e504577b3a676a82ae1631ec2340927
Sep 17 03:38:22.517775 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.517748    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/674d2736-fcb8-420e-b66a-dfd58993dcbf-maas-api-tls\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:38:22.517775 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.517772    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkv2j\" (UniqueName: \"kubernetes.io/projected/813bbf20-e4e8-4476-a390-e374a0e3c2db-kube-api-access-xkv2j\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:38:22.517955 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.517783    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c45rq\" (UniqueName: \"kubernetes.io/projected/674d2736-fcb8-420e-b66a-dfd58993dcbf-kube-api-access-c45rq\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:38:22.517955 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.517793    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8a47a4e-01e9-4fde-973d-8106d545b716-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:38:22.517955 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.517802    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/813bbf20-e4e8-4476-a390-e374a0e3c2db-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:38:22.517955 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.517810    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv6cq\" (UniqueName: \"kubernetes.io/projected/f8a47a4e-01e9-4fde-973d-8106d545b716-kube-api-access-hv6cq\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:38:22.517955 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.517818    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/674d2736-fcb8-420e-b66a-dfd58993dcbf-metrics-certs\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:38:22.643134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.643096    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:22.916004 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:22.915950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:23.259358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.259259    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg" event={"ID":"674d2736-fcb8-420e-b66a-dfd58993dcbf","Type":"ContainerDied","Data":"042383a1c38198df4b5c197b599e654b939bde2ab4cd9a5d97ffab031276ba1e"}
Sep 17 03:38:23.259358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.259289    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"
Sep 17 03:38:23.259873 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.259389    2576 scope.go:117] "RemoveContainer" containerID="9b9295f16dc002ccf18351349a21794051924aefbdb21b2012279273e827bdea"
Sep 17 03:38:23.261316 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.261286    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j" event={"ID":"1f9dd19b-fbaa-4266-b690-e45e3b636372","Type":"ContainerStarted","Data":"fbc0e8bb225b325ee3feedb90e73e8c9adb8f67ac180fb786d6231892b3d9567"}
Sep 17 03:38:23.261462 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.261352    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j" event={"ID":"1f9dd19b-fbaa-4266-b690-e45e3b636372","Type":"ContainerStarted","Data":"807693f9c0ee58e46254639d73644d377e504577b3a676a82ae1631ec2340927"}
Sep 17 03:38:23.261462 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.261398    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"
Sep 17 03:38:23.263014 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.262997    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"
Sep 17 03:38:23.263202 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.263005    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr" event={"ID":"813bbf20-e4e8-4476-a390-e374a0e3c2db","Type":"ContainerDied","Data":"59b8bdc48b6211111549a8cbed210858dcecda59a600a78c29efa807c64e0719"}
Sep 17 03:38:23.264576 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.264558    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62" event={"ID":"f8a47a4e-01e9-4fde-973d-8106d545b716","Type":"ContainerDied","Data":"59ec9780d6d27bdee18c73171090d9ba5eaafee1266cb2ff1fe095d1d0b71672"}
Sep 17 03:38:23.264689 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.264620    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"
Sep 17 03:38:23.270561 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.270493    2576 scope.go:117] "RemoveContainer" containerID="0d2540a5019665f5e91ba8dcf98f959d88eed853e3faa7888ba86fcaed046d78"
Sep 17 03:38:23.283410 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.283390    2576 scope.go:117] "RemoveContainer" containerID="827b7f67f8f39c0c250739ef609a14a9ad1e60cfba9e3a55b8d501a8037940c6"
Sep 17 03:38:23.291277 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.291225    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j" podStartSLOduration=2.291206102 podStartE2EDuration="2.291206102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:38:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:38:23.288768429 +0000 UTC m=+1131.633494509" watchObservedRunningTime="2026-09-17 03:38:23.291206102 +0000 UTC m=+1131.635932182"
Sep 17 03:38:23.301540 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.301506    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"]
Sep 17 03:38:23.308959 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.308935    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3d0c3c-6dc5f7488c-whrsg"]
Sep 17 03:38:23.318393 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.318357    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"]
Sep 17 03:38:23.320954 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.320922    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-3d0c3c-5fdddb8c5d-gzl62"]
Sep 17 03:38:23.328563 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.328536    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"]
Sep 17 03:38:23.334147 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.334123    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3d0c3c-f75b4b6dc-b9nsr"]
Sep 17 03:38:23.601766 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:23.601733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:24.294791 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:24.294757    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="674d2736-fcb8-420e-b66a-dfd58993dcbf" path="/var/lib/kubelet/pods/674d2736-fcb8-420e-b66a-dfd58993dcbf/volumes"
Sep 17 03:38:24.295148 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:24.295121    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="813bbf20-e4e8-4476-a390-e374a0e3c2db" path="/var/lib/kubelet/pods/813bbf20-e4e8-4476-a390-e374a0e3c2db/volumes"
Sep 17 03:38:24.295476 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:24.295460    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8a47a4e-01e9-4fde-973d-8106d545b716" path="/var/lib/kubelet/pods/f8a47a4e-01e9-4fde-973d-8106d545b716/volumes"
Sep 17 03:38:24.305148 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:24.305122    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:24.930982 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:24.930944    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:25.713344 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:25.713305    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:26.336448 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:26.336383    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:27.656158 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:27.656119    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:30.131803 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:30.131761    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:38:34.274995 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:34.274963    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"
Sep 17 03:38:43.111922 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:43.111882    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:43.798519 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:43.798481    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:44.433131 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:44.433094    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:45.404453 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:45.404396    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:45.505678 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:45.505631    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:46.141030 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:46.140985    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:46.800868 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:46.800831    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:46.913777 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:46.913745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:54.398704 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:54.398622    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:38:55.333227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:38:55.333185    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:04.484202 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:04.484163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:04.749502 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:04.749408    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:10.528178 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:10.528144    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:10.595331 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:10.595293    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:10.803574 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:10.803543    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:11.380949 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:11.380920    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:11.699910 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:11.699796    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:12.019685 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:12.019598    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:14.895645 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:14.895584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:15.121332 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:15.121290    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:15.208184 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:15.208094    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:15.600499 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:15.600471    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:16.247866 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:16.247830    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:16.406970 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:16.406930    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:17.124277 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:17.124235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:17.765911 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:17.765871    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:18.485896 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.485861    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"]
Sep 17 03:39:18.487061 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.487033    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="813bbf20-e4e8-4476-a390-e374a0e3c2db" containerName="payload-processing"
Sep 17 03:39:18.487219 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.487206    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="813bbf20-e4e8-4476-a390-e374a0e3c2db" containerName="payload-processing"
Sep 17 03:39:18.487311 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.487301    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8a47a4e-01e9-4fde-973d-8106d545b716" containerName="payload-pre-processing"
Sep 17 03:39:18.487388 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.487378    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8a47a4e-01e9-4fde-973d-8106d545b716" containerName="payload-pre-processing"
Sep 17 03:39:18.487506 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.487495    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="674d2736-fcb8-420e-b66a-dfd58993dcbf" containerName="maas-api"
Sep 17 03:39:18.487594 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.487585    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="674d2736-fcb8-420e-b66a-dfd58993dcbf" containerName="maas-api"
Sep 17 03:39:18.487893 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.487880    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="813bbf20-e4e8-4476-a390-e374a0e3c2db" containerName="payload-processing"
Sep 17 03:39:18.487990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.487981    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="674d2736-fcb8-420e-b66a-dfd58993dcbf" containerName="maas-api"
Sep 17 03:39:18.488071 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.488062    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8a47a4e-01e9-4fde-973d-8106d545b716" containerName="payload-pre-processing"
Sep 17 03:39:18.493966 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.493942    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:18.498669 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.498645    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-22e045-serving-cert\""
Sep 17 03:39:18.503552 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.503528    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"]
Sep 17 03:39:18.522041 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.522013    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:18.526295 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.526264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhdj7\" (UniqueName: \"kubernetes.io/projected/fc41ede3-20c0-40b8-a131-f8979f03a1e1-kube-api-access-dhdj7\") pod \"maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:18.526453 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.526326    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:18.526453 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.526393    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:18.526453 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.526414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc41ede3-20c0-40b8-a131-f8979f03a1e1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:18.623154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.623126    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:18.627021 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.626993    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:18.627170 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.627050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:18.627170 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.627074    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc41ede3-20c0-40b8-a131-f8979f03a1e1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:18.627170 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.627108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhdj7\" (UniqueName: \"kubernetes.io/projected/fc41ede3-20c0-40b8-a131-f8979f03a1e1-kube-api-access-dhdj7\") pod \"maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:18.627291 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:39:18.627170    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-serving-cert: secret "maas-api-e2e-shared-a-w3-22e045-serving-cert" not found
Sep 17 03:39:18.627291 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:39:18.627259    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-maas-api-tls podName:fc41ede3-20c0-40b8-a131-f8979f03a1e1 nodeName:}" failed. No retries permitted until 2026-09-17 03:39:19.127238855 +0000 UTC m=+1187.471964937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-maas-api-tls") pod "maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh" (UID: "fc41ede3-20c0-40b8-a131-f8979f03a1e1") : secret "maas-api-e2e-shared-a-w3-22e045-serving-cert" not found
Sep 17 03:39:18.627838 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.627813    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc41ede3-20c0-40b8-a131-f8979f03a1e1-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:18.629316 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.629294    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:18.634826 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.634800    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhdj7\" (UniqueName: \"kubernetes.io/projected/fc41ede3-20c0-40b8-a131-f8979f03a1e1-kube-api-access-dhdj7\") pod \"maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:18.821713 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.821683    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:18.968273 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:18.968240    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:19.067343 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.067287    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:19.106896 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.106811    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"]
Sep 17 03:39:19.110511 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.110495    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"
Sep 17 03:39:19.112582 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.112560    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-22e045-dockercfg-ckvr8\""
Sep 17 03:39:19.112706 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.112564    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-22e045\""
Sep 17 03:39:19.118695 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.118662    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"]
Sep 17 03:39:19.132385 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.132357    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w6q8\" (UniqueName: \"kubernetes.io/projected/941c982b-fa6f-4f6f-b088-589bf23e7162-kube-api-access-7w6q8\") pod \"payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7\" (UID: \"941c982b-fa6f-4f6f-b088-589bf23e7162\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"
Sep 17 03:39:19.132584 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.132393    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/941c982b-fa6f-4f6f-b088-589bf23e7162-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7\" (UID: \"941c982b-fa6f-4f6f-b088-589bf23e7162\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"
Sep 17 03:39:19.132584 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.132473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:19.134930 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.134906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:19.140493 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.140464    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"]
Sep 17 03:39:19.143993 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.143974    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"
Sep 17 03:39:19.153865 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.153838    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"]
Sep 17 03:39:19.233097 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.233056    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w6q8\" (UniqueName: \"kubernetes.io/projected/941c982b-fa6f-4f6f-b088-589bf23e7162-kube-api-access-7w6q8\") pod \"payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7\" (UID: \"941c982b-fa6f-4f6f-b088-589bf23e7162\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"
Sep 17 03:39:19.233097 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.233096    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/941c982b-fa6f-4f6f-b088-589bf23e7162-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7\" (UID: \"941c982b-fa6f-4f6f-b088-589bf23e7162\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"
Sep 17 03:39:19.233311 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.233143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f7764ac-5295-4921-aa0c-2e132638125c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv\" (UID: \"3f7764ac-5295-4921-aa0c-2e132638125c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"
Sep 17 03:39:19.233311 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.233188    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77tsq\" (UniqueName: \"kubernetes.io/projected/3f7764ac-5295-4921-aa0c-2e132638125c-kube-api-access-77tsq\") pod \"payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv\" (UID: \"3f7764ac-5295-4921-aa0c-2e132638125c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"
Sep 17 03:39:19.233877 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.233857    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/941c982b-fa6f-4f6f-b088-589bf23e7162-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7\" (UID: \"941c982b-fa6f-4f6f-b088-589bf23e7162\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"
Sep 17 03:39:19.239968 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.239944    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w6q8\" (UniqueName: \"kubernetes.io/projected/941c982b-fa6f-4f6f-b088-589bf23e7162-kube-api-access-7w6q8\") pod \"payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7\" (UID: \"941c982b-fa6f-4f6f-b088-589bf23e7162\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"
Sep 17 03:39:19.334077 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.334035    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f7764ac-5295-4921-aa0c-2e132638125c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv\" (UID: \"3f7764ac-5295-4921-aa0c-2e132638125c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"
Sep 17 03:39:19.334273 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.334100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77tsq\" (UniqueName: \"kubernetes.io/projected/3f7764ac-5295-4921-aa0c-2e132638125c-kube-api-access-77tsq\") pod \"payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv\" (UID: \"3f7764ac-5295-4921-aa0c-2e132638125c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"
Sep 17 03:39:19.334828 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.334796    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f7764ac-5295-4921-aa0c-2e132638125c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv\" (UID: \"3f7764ac-5295-4921-aa0c-2e132638125c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"
Sep 17 03:39:19.341937 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.341910    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77tsq\" (UniqueName: \"kubernetes.io/projected/3f7764ac-5295-4921-aa0c-2e132638125c-kube-api-access-77tsq\") pod \"payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv\" (UID: \"3f7764ac-5295-4921-aa0c-2e132638125c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"
Sep 17 03:39:19.409315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.409231    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:19.423606 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.423580    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"
Sep 17 03:39:19.454766 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.454728    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"
Sep 17 03:39:19.683909 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.683877    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"]
Sep 17 03:39:19.685289 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:39:19.685257    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc41ede3_20c0_40b8_a131_f8979f03a1e1.slice/crio-aef50505a79301c20639273c9583b3c0718b9ae29ed8233510f0dfb3d386a874 WatchSource:0}: Error finding container aef50505a79301c20639273c9583b3c0718b9ae29ed8233510f0dfb3d386a874: Status 404 returned error can't find the container with id aef50505a79301c20639273c9583b3c0718b9ae29ed8233510f0dfb3d386a874
Sep 17 03:39:19.706038 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.706014    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"]
Sep 17 03:39:19.707519 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:39:19.707495    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod941c982b_fa6f_4f6f_b088_589bf23e7162.slice/crio-0e7ee66987880bf00a016703a7b937351b8111f5cd1b362141a6c898e5827f06 WatchSource:0}: Error finding container 0e7ee66987880bf00a016703a7b937351b8111f5cd1b362141a6c898e5827f06: Status 404 returned error can't find the container with id 0e7ee66987880bf00a016703a7b937351b8111f5cd1b362141a6c898e5827f06
Sep 17 03:39:19.755014 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:19.754986    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"]
Sep 17 03:39:19.756180 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:39:19.756155    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f7764ac_5295_4921_aa0c_2e132638125c.slice/crio-ba7041dd2bf463b36662a028d75eaa7300b58caa8caedf6b3bc413996bb346a7 WatchSource:0}: Error finding container ba7041dd2bf463b36662a028d75eaa7300b58caa8caedf6b3bc413996bb346a7: Status 404 returned error can't find the container with id ba7041dd2bf463b36662a028d75eaa7300b58caa8caedf6b3bc413996bb346a7
Sep 17 03:39:20.601559 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:20.601515    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7" event={"ID":"941c982b-fa6f-4f6f-b088-589bf23e7162","Type":"ContainerStarted","Data":"5ca225bf0332935212d5d3b652130cdf33abd050b4d7ade884e1fc1e5ae82512"}
Sep 17 03:39:20.602054 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:20.601588    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7" event={"ID":"941c982b-fa6f-4f6f-b088-589bf23e7162","Type":"ContainerStarted","Data":"0e7ee66987880bf00a016703a7b937351b8111f5cd1b362141a6c898e5827f06"}
Sep 17 03:39:20.602054 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:20.601626    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"
Sep 17 03:39:20.603328 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:20.603297    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv" event={"ID":"3f7764ac-5295-4921-aa0c-2e132638125c","Type":"ContainerStarted","Data":"b64c11fb315bcfd42cb83c284b4bcaa758951274e917ab740b702d4b691d5a61"}
Sep 17 03:39:20.603480 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:20.603358    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv" event={"ID":"3f7764ac-5295-4921-aa0c-2e132638125c","Type":"ContainerStarted","Data":"ba7041dd2bf463b36662a028d75eaa7300b58caa8caedf6b3bc413996bb346a7"}
Sep 17 03:39:20.603480 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:20.603392    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"
Sep 17 03:39:20.605050 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:20.605025    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh" event={"ID":"fc41ede3-20c0-40b8-a131-f8979f03a1e1","Type":"ContainerStarted","Data":"1293da3aaa0b7e209dcbab3cb07f88dbb972e37e87a26af4717e325d59f51ed3"}
Sep 17 03:39:20.605159 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:20.605073    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh" event={"ID":"fc41ede3-20c0-40b8-a131-f8979f03a1e1","Type":"ContainerStarted","Data":"aef50505a79301c20639273c9583b3c0718b9ae29ed8233510f0dfb3d386a874"}
Sep 17 03:39:20.605216 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:20.605161    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:20.618084 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:20.618009    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7" podStartSLOduration=1.617993792 podStartE2EDuration="1.617993792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:39:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:39:20.61555403 +0000 UTC m=+1188.960280109" watchObservedRunningTime="2026-09-17 03:39:20.617993792 +0000 UTC m=+1188.962719871"
Sep 17 03:39:20.630567 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:20.630521    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv" podStartSLOduration=1.630505571 podStartE2EDuration="1.630505571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:39:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:39:20.628823428 +0000 UTC m=+1188.973549509" watchObservedRunningTime="2026-09-17 03:39:20.630505571 +0000 UTC m=+1188.975231688"
Sep 17 03:39:20.644083 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:20.644034    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh" podStartSLOduration=2.183344268 podStartE2EDuration="2.644017548s" totalImagesPullingTime="460.67328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:39:20.643034207 +0000 UTC m=+1188.987760287" watchObservedRunningTime="2026-09-17 03:39:20.644017548 +0000 UTC m=+1188.988743627"
Sep 17 03:39:22.167157 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:22.167125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:26.340284 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:26.340248    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:39:26.617902 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:26.617826    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:39:28.528824 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:28.528790    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:28.745127 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:28.745093    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:28.848153 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:28.848115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:31.614400 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:31.614367    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"
Sep 17 03:39:31.614822 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:31.614466    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"
Sep 17 03:39:32.267724 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:32.267691    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:39:32.267900 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:32.267788    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:39:33.872939 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:33.872903    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:40.069127 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:40.069076    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:40.385361 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:40.385266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:40.522165 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:40.522123    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:40.679494 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:40.679386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:41.286495 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:41.286459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:41.384722 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:41.384691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:42.015138 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:42.015102    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:43.215566 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:43.215530    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:43.320794 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:43.320754    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:43.419691 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:43.419647    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:43.718763 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:43.718729    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:44.414710 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:44.414677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:45.400930 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:45.400902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:46.112016 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:46.111975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:46.212156 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:46.212115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:46.400113 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:46.400013    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:46.660469 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:46.660354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:46.759591 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:46.759552    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:47.071318 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:47.071289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:47.806810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:47.806776    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:48.484918 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:48.484891    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:49.234078 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:49.234044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:39:58.735613 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:39:58.735579    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:40:00.482229 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:00.482197    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:04.586421 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:04.586381    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:04.970260 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:04.970173    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"]
Sep 17 03:40:04.990793 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:04.990756    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:08.192906 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:08.192866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:08.293642 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:08.293609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:08.922796 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:08.922766    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:09.586867 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:09.586834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:10.314664 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:10.314633    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:11.016583 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:11.016552    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:11.253049 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:11.253018    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:11.826279 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:11.826248    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:12.665039 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:12.665006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:13.418374 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:13.418335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:19.997118 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:19.997078    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5" podUID="d1a71d6c-6eb1-4657-a150-02ecdeacff77" containerName="main" containerID="cri-o://d3287c1e78420dbcecf2e20abb2148dc93e423d05c4748d8411a2fa5cbb81aa5" gracePeriod=45
Sep 17 03:40:20.226587 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.226553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:20.261895 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.261838    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:40:20.408065 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.408028    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-tmp-dir\") pod \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") "
Sep 17 03:40:20.408065 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.408070    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1a71d6c-6eb1-4657-a150-02ecdeacff77-tls-certs\") pod \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") "
Sep 17 03:40:20.408320 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.408085    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-home\") pod \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") "
Sep 17 03:40:20.408320 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.408122    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjsmn\" (UniqueName: \"kubernetes.io/projected/d1a71d6c-6eb1-4657-a150-02ecdeacff77-kube-api-access-qjsmn\") pod \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") "
Sep 17 03:40:20.408320 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.408179    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-dshm\") pod \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") "
Sep 17 03:40:20.408320 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.408200    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-model-cache\") pod \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\" (UID: \"d1a71d6c-6eb1-4657-a150-02ecdeacff77\") "
Sep 17 03:40:20.408633 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.408380    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-home" (OuterVolumeSpecName: "home") pod "d1a71d6c-6eb1-4657-a150-02ecdeacff77" (UID: "d1a71d6c-6eb1-4657-a150-02ecdeacff77"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:40:20.408633 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.408392    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d1a71d6c-6eb1-4657-a150-02ecdeacff77" (UID: "d1a71d6c-6eb1-4657-a150-02ecdeacff77"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:40:20.408633 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.408573    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-model-cache" (OuterVolumeSpecName: "model-cache") pod "d1a71d6c-6eb1-4657-a150-02ecdeacff77" (UID: "d1a71d6c-6eb1-4657-a150-02ecdeacff77"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:40:20.410468 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.410424    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-dshm" (OuterVolumeSpecName: "dshm") pod "d1a71d6c-6eb1-4657-a150-02ecdeacff77" (UID: "d1a71d6c-6eb1-4657-a150-02ecdeacff77"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:40:20.410544 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.410461    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1a71d6c-6eb1-4657-a150-02ecdeacff77-kube-api-access-qjsmn" (OuterVolumeSpecName: "kube-api-access-qjsmn") pod "d1a71d6c-6eb1-4657-a150-02ecdeacff77" (UID: "d1a71d6c-6eb1-4657-a150-02ecdeacff77"). InnerVolumeSpecName "kube-api-access-qjsmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:40:20.410581 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.410547    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1a71d6c-6eb1-4657-a150-02ecdeacff77-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d1a71d6c-6eb1-4657-a150-02ecdeacff77" (UID: "d1a71d6c-6eb1-4657-a150-02ecdeacff77"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:40:20.509272 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.509237    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-dshm\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:40:20.509272 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.509262    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-model-cache\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:40:20.509272 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.509272    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-tmp-dir\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:40:20.509272 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.509280    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1a71d6c-6eb1-4657-a150-02ecdeacff77-tls-certs\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:40:20.509620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.509287    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1a71d6c-6eb1-4657-a150-02ecdeacff77-home\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:40:20.509620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.509308    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjsmn\" (UniqueName: \"kubernetes.io/projected/d1a71d6c-6eb1-4657-a150-02ecdeacff77-kube-api-access-qjsmn\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:40:20.964896 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.964858    2576 generic.go:356] "Generic (PLEG): container finished" podID="d1a71d6c-6eb1-4657-a150-02ecdeacff77" containerID="d3287c1e78420dbcecf2e20abb2148dc93e423d05c4748d8411a2fa5cbb81aa5" exitCode=0
Sep 17 03:40:20.965073 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.964912    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5" event={"ID":"d1a71d6c-6eb1-4657-a150-02ecdeacff77","Type":"ContainerDied","Data":"d3287c1e78420dbcecf2e20abb2148dc93e423d05c4748d8411a2fa5cbb81aa5"}
Sep 17 03:40:20.965073 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.964920    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"
Sep 17 03:40:20.965073 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.964943    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5" event={"ID":"d1a71d6c-6eb1-4657-a150-02ecdeacff77","Type":"ContainerDied","Data":"ad2d3929de58f86b8fe842c3233fe831f87d911fd55b7f006e1e8cd0d7be9b59"}
Sep 17 03:40:20.965073 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.964958    2576 scope.go:117] "RemoveContainer" containerID="d3287c1e78420dbcecf2e20abb2148dc93e423d05c4748d8411a2fa5cbb81aa5"
Sep 17 03:40:20.974551 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.974530    2576 scope.go:117] "RemoveContainer" containerID="d3287c1e78420dbcecf2e20abb2148dc93e423d05c4748d8411a2fa5cbb81aa5"
Sep 17 03:40:20.974847 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:40:20.974828    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3287c1e78420dbcecf2e20abb2148dc93e423d05c4748d8411a2fa5cbb81aa5\": container with ID starting with d3287c1e78420dbcecf2e20abb2148dc93e423d05c4748d8411a2fa5cbb81aa5 not found: ID does not exist" containerID="d3287c1e78420dbcecf2e20abb2148dc93e423d05c4748d8411a2fa5cbb81aa5"
Sep 17 03:40:20.974903 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.974856    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3287c1e78420dbcecf2e20abb2148dc93e423d05c4748d8411a2fa5cbb81aa5"} err="failed to get container status \"d3287c1e78420dbcecf2e20abb2148dc93e423d05c4748d8411a2fa5cbb81aa5\": rpc error: code = NotFound desc = could not find container \"d3287c1e78420dbcecf2e20abb2148dc93e423d05c4748d8411a2fa5cbb81aa5\": container with ID starting with d3287c1e78420dbcecf2e20abb2148dc93e423d05c4748d8411a2fa5cbb81aa5 not found: ID does not exist"
Sep 17 03:40:20.986489 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.986454    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"]
Sep 17 03:40:20.995743 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:20.995711    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/e2e-conflict-b-fed91830-kserve-6c9687b4c9-srvm5"]
Sep 17 03:40:22.294340 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:22.294295    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1a71d6c-6eb1-4657-a150-02ecdeacff77" path="/var/lib/kubelet/pods/d1a71d6c-6eb1-4657-a150-02ecdeacff77/volumes"
Sep 17 03:40:32.141058 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:32.140972    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:40:35.588763 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.588729    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"]
Sep 17 03:40:35.589216 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.589151    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1a71d6c-6eb1-4657-a150-02ecdeacff77" containerName="main"
Sep 17 03:40:35.589216 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.589162    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1a71d6c-6eb1-4657-a150-02ecdeacff77" containerName="main"
Sep 17 03:40:35.589352 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.589237    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1a71d6c-6eb1-4657-a150-02ecdeacff77" containerName="main"
Sep 17 03:40:35.593676 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.593654    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.596680 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.596649    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-22e045\"/\"kube-root-ca.crt\""
Sep 17 03:40:35.596680 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.596647    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-22e045\"/\"openshift-service-ca.crt\""
Sep 17 03:40:35.596889 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.596653    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-22e045\"/\"default-dockercfg-8dd44\""
Sep 17 03:40:35.596889 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.596739    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-22e045\"/\"auth-test-model-22e045-kserve-self-signed-certs\""
Sep 17 03:40:35.603811 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.603789    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"]
Sep 17 03:40:35.627654 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.627625    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:35.747022 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.746986    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-dshm\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.747228 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.747046    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-tmp-dir\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.747228 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.747100    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-model-cache\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.747228 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.747174    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7djxw\" (UniqueName: \"kubernetes.io/projected/65348adf-214b-4933-a984-431523fa7b21-kube-api-access-7djxw\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.747228 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.747211    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-home\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.747420 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.747280    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65348adf-214b-4933-a984-431523fa7b21-tls-certs\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.848671 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.848581    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65348adf-214b-4933-a984-431523fa7b21-tls-certs\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.848671 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.848636    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-dshm\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.848891 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.848676    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-tmp-dir\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.848891 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.848706    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-model-cache\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.848891 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.848766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7djxw\" (UniqueName: \"kubernetes.io/projected/65348adf-214b-4933-a984-431523fa7b21-kube-api-access-7djxw\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.848891 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.848806    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-home\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.849159 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.849124    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-tmp-dir\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.849159 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.849157    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-model-cache\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.849317 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.849181    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-home\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.851009 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.850983    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-dshm\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.851141 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.851126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65348adf-214b-4933-a984-431523fa7b21-tls-certs\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.856113 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.856093    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7djxw\" (UniqueName: \"kubernetes.io/projected/65348adf-214b-4933-a984-431523fa7b21-kube-api-access-7djxw\") pod \"auth-test-model-22e045-kserve-67ddf895ff-97rpm\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") " pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:35.904383 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:35.904354    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:36.179160 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:36.179128    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"]
Sep 17 03:40:36.180848 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:40:36.180821    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65348adf_214b_4933_a984_431523fa7b21.slice/crio-9a2c90d2aaa682943f48337ac5f58c13fe87ff04eca92344382d9316d21a51fb WatchSource:0}: Error finding container 9a2c90d2aaa682943f48337ac5f58c13fe87ff04eca92344382d9316d21a51fb: Status 404 returned error can't find the container with id 9a2c90d2aaa682943f48337ac5f58c13fe87ff04eca92344382d9316d21a51fb
Sep 17 03:40:37.056561 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:37.056520    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm" event={"ID":"65348adf-214b-4933-a984-431523fa7b21","Type":"ContainerStarted","Data":"3d94687287fa5eb86ab637e2178c904461500b00ac8e3d2ab532e414016763ec"}
Sep 17 03:40:37.056960 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:37.056571    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm" event={"ID":"65348adf-214b-4933-a984-431523fa7b21","Type":"ContainerStarted","Data":"9a2c90d2aaa682943f48337ac5f58c13fe87ff04eca92344382d9316d21a51fb"}
Sep 17 03:40:37.074319 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:37.074265    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm" podStartSLOduration=2.074236739 podStartE2EDuration="2.074236739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:40:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:40:37.072318678 +0000 UTC m=+1265.417044757" watchObservedRunningTime="2026-09-17 03:40:37.074236739 +0000 UTC m=+1265.418962818"
Sep 17 03:40:45.904655 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:45.904598    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:45.904655 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:45.904670    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:45.917470 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:45.917442    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:46.121162 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:46.121129    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:40:51.731978 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:51.731942    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:53.518899 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:53.518862    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:54.416983 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:54.416948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:55.188549 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.188503    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"]
Sep 17 03:40:55.193798 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.193769    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.194964 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.194938    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"]
Sep 17 03:40:55.195901 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.195883    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-214fea\"/\"openshift-service-ca.crt\""
Sep 17 03:40:55.196619 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.196590    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-214fea\"/\"kube-root-ca.crt\""
Sep 17 03:40:55.196744 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.196632    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-214fea\"/\"distinct-w4-214fea-kserve-self-signed-certs\""
Sep 17 03:40:55.196821 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.196797    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-214fea\"/\"default-dockercfg-g7kkn\""
Sep 17 03:40:55.223089 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.223057    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-tmp-dir\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.223208 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.223128    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2247v\" (UniqueName: \"kubernetes.io/projected/7c845dae-d21b-48c3-a45a-81eaecab1847-kube-api-access-2247v\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.223208 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.223171    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c845dae-d21b-48c3-a45a-81eaecab1847-tls-certs\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.223208 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.223198    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-dshm\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.223334 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.223215    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-model-cache\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.223334 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.223237    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-home\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.263751 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.263714    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:55.324131 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.324091    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-tmp-dir\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.324340 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.324165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2247v\" (UniqueName: \"kubernetes.io/projected/7c845dae-d21b-48c3-a45a-81eaecab1847-kube-api-access-2247v\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.324340 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.324190    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c845dae-d21b-48c3-a45a-81eaecab1847-tls-certs\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.324340 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.324209    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-dshm\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.324340 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.324224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-model-cache\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.324340 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.324243    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-home\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.324680 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.324625    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-tmp-dir\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.324680 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.324668    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-model-cache\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.324758 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.324713    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-home\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.326498 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.326473    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-dshm\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.326693 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.326678    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c845dae-d21b-48c3-a45a-81eaecab1847-tls-certs\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.332982 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.332959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2247v\" (UniqueName: \"kubernetes.io/projected/7c845dae-d21b-48c3-a45a-81eaecab1847-kube-api-access-2247v\") pod \"distinct-w4-214fea-kserve-77f49b7446-4p4p5\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.365840 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.365781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:55.506828 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.506753    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:40:55.781633 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:55.781606    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"]
Sep 17 03:40:55.783706 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:40:55.783675    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c845dae_d21b_48c3_a45a_81eaecab1847.slice/crio-8bccf3b4f2add5808cba20ed21576bb3f57e9cbe666a0b4aa1f638f109033eb7 WatchSource:0}: Error finding container 8bccf3b4f2add5808cba20ed21576bb3f57e9cbe666a0b4aa1f638f109033eb7: Status 404 returned error can't find the container with id 8bccf3b4f2add5808cba20ed21576bb3f57e9cbe666a0b4aa1f638f109033eb7
Sep 17 03:40:56.167180 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:56.167148    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5" event={"ID":"7c845dae-d21b-48c3-a45a-81eaecab1847","Type":"ContainerStarted","Data":"e19b8a7603f55b6559e94626fa7835cbc850f0ca00a2eb622db4579c9ae2c997"}
Sep 17 03:40:56.167351 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:56.167194    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5" event={"ID":"7c845dae-d21b-48c3-a45a-81eaecab1847","Type":"ContainerStarted","Data":"8bccf3b4f2add5808cba20ed21576bb3f57e9cbe666a0b4aa1f638f109033eb7"}
Sep 17 03:40:56.185787 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:56.185736    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5" podStartSLOduration=1.185720721 podStartE2EDuration="1.185720721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:40:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:40:56.18247863 +0000 UTC m=+1284.527204707" watchObservedRunningTime="2026-09-17 03:40:56.185720721 +0000 UTC m=+1284.530446796"
Sep 17 03:40:56.191182 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:56.191150    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:56.861971 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:56.861940    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:57.575738 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:57.575703    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:58.414619 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:58.414582    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:40:59.102140 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:40:59.102105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:00.011037 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:00.010968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:03.939958 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:03.939915    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:41:05.506907 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:05.506867    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:41:05.506907 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:05.506916    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:41:05.519233 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:05.519203    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:41:06.244005 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:06.243977    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:41:08.974458 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:08.974407    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:09.003570 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.003540    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"]
Sep 17 03:41:09.003782 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.003756    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j" podUID="1f9dd19b-fbaa-4266-b690-e45e3b636372" containerName="payload-processing" containerID="cri-o://fbc0e8bb225b325ee3feedb90e73e8c9adb8f67ac180fb786d6231892b3d9567" gracePeriod=30
Sep 17 03:41:09.071995 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.071957    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:09.159664 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.159631    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:09.257059 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.257031    2576 generic.go:356] "Generic (PLEG): container finished" podID="1f9dd19b-fbaa-4266-b690-e45e3b636372" containerID="fbc0e8bb225b325ee3feedb90e73e8c9adb8f67ac180fb786d6231892b3d9567" exitCode=0
Sep 17 03:41:09.257204 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.257117    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j" event={"ID":"1f9dd19b-fbaa-4266-b690-e45e3b636372","Type":"ContainerDied","Data":"fbc0e8bb225b325ee3feedb90e73e8c9adb8f67ac180fb786d6231892b3d9567"}
Sep 17 03:41:09.257204 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.257185    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j" event={"ID":"1f9dd19b-fbaa-4266-b690-e45e3b636372","Type":"ContainerDied","Data":"807693f9c0ee58e46254639d73644d377e504577b3a676a82ae1631ec2340927"}
Sep 17 03:41:09.257295 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.257209    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="807693f9c0ee58e46254639d73644d377e504577b3a676a82ae1631ec2340927"
Sep 17 03:41:09.258299 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.258273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:09.271152 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.271134    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"
Sep 17 03:41:09.330092 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.330047    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:09.356208 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.356172    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w65dw\" (UniqueName: \"kubernetes.io/projected/1f9dd19b-fbaa-4266-b690-e45e3b636372-kube-api-access-w65dw\") pod \"1f9dd19b-fbaa-4266-b690-e45e3b636372\" (UID: \"1f9dd19b-fbaa-4266-b690-e45e3b636372\") "
Sep 17 03:41:09.356360 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.356347    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f9dd19b-fbaa-4266-b690-e45e3b636372-plugins-config-volume\") pod \"1f9dd19b-fbaa-4266-b690-e45e3b636372\" (UID: \"1f9dd19b-fbaa-4266-b690-e45e3b636372\") "
Sep 17 03:41:09.356731 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.356705    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f9dd19b-fbaa-4266-b690-e45e3b636372-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1f9dd19b-fbaa-4266-b690-e45e3b636372" (UID: "1f9dd19b-fbaa-4266-b690-e45e3b636372"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:41:09.358471 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.358425    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9dd19b-fbaa-4266-b690-e45e3b636372-kube-api-access-w65dw" (OuterVolumeSpecName: "kube-api-access-w65dw") pod "1f9dd19b-fbaa-4266-b690-e45e3b636372" (UID: "1f9dd19b-fbaa-4266-b690-e45e3b636372"). InnerVolumeSpecName "kube-api-access-w65dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:41:09.457673 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.457640    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f9dd19b-fbaa-4266-b690-e45e3b636372-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:41:09.457673 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.457666    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w65dw\" (UniqueName: \"kubernetes.io/projected/1f9dd19b-fbaa-4266-b690-e45e3b636372-kube-api-access-w65dw\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:41:09.708820 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:09.708782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:10.262768 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:10.262735    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"
Sep 17 03:41:10.284404 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:10.284378    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"]
Sep 17 03:41:10.294903 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:10.294879    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c10ab8fb-67d9d76657-q5g5j"]
Sep 17 03:41:10.492214 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:10.492165    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:11.307278 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:11.307244    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:12.295988 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:12.295953    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f9dd19b-fbaa-4266-b690-e45e3b636372" path="/var/lib/kubelet/pods/1f9dd19b-fbaa-4266-b690-e45e3b636372/volumes"
Sep 17 03:41:12.309551 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:12.309521    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:12.447336 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:12.447306    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:13.048371 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:13.048340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:13.812765 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:13.812724    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:14.740775 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:14.740742    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:17.480071 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:17.480040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:35.746420 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:35.746383    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:41:36.909107 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:36.909078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:47.081022 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.080983    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"]
Sep 17 03:41:47.081683 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.081661    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9dd19b-fbaa-4266-b690-e45e3b636372" containerName="payload-processing"
Sep 17 03:41:47.081736 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.081689    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9dd19b-fbaa-4266-b690-e45e3b636372" containerName="payload-processing"
Sep 17 03:41:47.081799 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.081786    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9dd19b-fbaa-4266-b690-e45e3b636372" containerName="payload-processing"
Sep 17 03:41:47.085942 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.085917    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.088316 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.088284    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-214fea\"/\"distinct-2-w4-214fea-kserve-self-signed-certs\""
Sep 17 03:41:47.094028 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.093998    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"]
Sep 17 03:41:47.114949 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.114923    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:47.203485 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.203450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-tmp-dir\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.203671 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.203503    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-model-cache\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.203671 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.203528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-home\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.203671 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.203613    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-tls-certs\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.203671 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.203652    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc988\" (UniqueName: \"kubernetes.io/projected/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-kube-api-access-sc988\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.203878 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.203764    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-dshm\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.304950 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.304919    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-tmp-dir\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.305130 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.304970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-model-cache\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.305130 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.305083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-home\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.305225 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.305141    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-tls-certs\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.305225 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.305180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc988\" (UniqueName: \"kubernetes.io/projected/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-kube-api-access-sc988\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.305318 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.305264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-dshm\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.305377 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.305311    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-model-cache\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.305377 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.305350    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-tmp-dir\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.305508 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.305401    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-home\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.307482 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.307457    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-dshm\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.307741 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.307724    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-tls-certs\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.313067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.313038    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc988\" (UniqueName: \"kubernetes.io/projected/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-kube-api-access-sc988\") pod \"distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") " pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.397685 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.397597    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:47.669081 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:47.669057    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"]
Sep 17 03:41:47.670778 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:41:47.670744    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8db5a26f_43bd_4c0a_88cd_630d7ebcac64.slice/crio-2463c27d1305920d143d3d73b0e53c1e086d1d257de86142ce19a76a107f1f74 WatchSource:0}: Error finding container 2463c27d1305920d143d3d73b0e53c1e086d1d257de86142ce19a76a107f1f74: Status 404 returned error can't find the container with id 2463c27d1305920d143d3d73b0e53c1e086d1d257de86142ce19a76a107f1f74
Sep 17 03:41:48.497372 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:48.497322    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb" event={"ID":"8db5a26f-43bd-4c0a-88cd-630d7ebcac64","Type":"ContainerStarted","Data":"11af2ebaa64e31f6f883cb2a16c67fb36aab6bea03c8bae03e4e650f6125b52b"}
Sep 17 03:41:48.497790 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:48.497382    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb" event={"ID":"8db5a26f-43bd-4c0a-88cd-630d7ebcac64","Type":"ContainerStarted","Data":"2463c27d1305920d143d3d73b0e53c1e086d1d257de86142ce19a76a107f1f74"}
Sep 17 03:41:54.270685 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:54.270603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:54.755623 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:54.755589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:41:57.398170 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:57.398135    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:57.398581 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:57.398374    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:57.410786 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:57.410753    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:41:57.428808 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:57.428761    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb" podStartSLOduration=10.428744281 podStartE2EDuration="10.428744281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:41:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:41:48.514171287 +0000 UTC m=+1336.858897380" watchObservedRunningTime="2026-09-17 03:41:57.428744281 +0000 UTC m=+1345.773470397"
Sep 17 03:41:57.568004 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:41:57.567978    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:42:00.689393 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:00.689361    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:14.200021 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:14.199971    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:14.810015 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:14.809981    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:14.871298 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:14.871260    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:15.077578 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:15.077482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:15.308848 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:15.308811    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:16.297490 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:16.297453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:16.386720 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:16.386677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:16.487842 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:16.487806    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:17.061443 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:17.061399    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:17.154202 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:17.154166    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:17.253687 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:17.253654    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:18.000160 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:18.000118    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:42:18.476174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:18.476141    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:24.246929 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:24.246893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:24.314328 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:24.314292    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:24.921585 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:24.921552    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:29.603840 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:29.603796    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:30.721207 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:30.721177    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:30.823525 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:30.823490    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:30.919762 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:30.919675    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:32.811608 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:32.811570    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:39.998217 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:39.998184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:40.053225 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:40.053190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:41.431587 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:41.431545    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:41.633511 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:41.633482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:42:53.150163 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:53.150125    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:42:55.252280 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:42:55.252235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:02.555178 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:02.555124    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:04.874488 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:04.874450    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:08.616019 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:08.615983    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:08.683467 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:08.683366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:08.936737 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:08.936646    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:08.978421 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:08.978389    2576 generic.go:356] "Generic (PLEG): container finished" podID="b16dffd3-95d8-4fa6-ac56-fb30275e4c24" containerID="59f827834ae9772fa54f3b9bad7d4ad30fb78d7760c2b082c84a3f9d658fe595" exitCode=137
Sep 17 03:43:08.978573 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:08.978465    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx" event={"ID":"b16dffd3-95d8-4fa6-ac56-fb30275e4c24","Type":"ContainerDied","Data":"59f827834ae9772fa54f3b9bad7d4ad30fb78d7760c2b082c84a3f9d658fe595"}
Sep 17 03:43:08.978819 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:08.978806    2576 scope.go:117] "RemoveContainer" containerID="59f827834ae9772fa54f3b9bad7d4ad30fb78d7760c2b082c84a3f9d658fe595"
Sep 17 03:43:09.289374 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:09.289277    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:09.401976 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:09.401944    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:43:09.987016 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:09.986983    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx" event={"ID":"b16dffd3-95d8-4fa6-ac56-fb30275e4c24","Type":"ContainerStarted","Data":"1e5050bd92532cc447bc5ec43d834236cae44b5a30efe99b961c3470ec0c5f14"}
Sep 17 03:43:09.987421 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:09.987079    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:43:10.175574 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:10.175541    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:11.182409 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:11.182368    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:11.916900 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:11.916867    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:12.020258 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:12.020225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:12.133553 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:12.133519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:13.019887 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:13.019852    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:13.877753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:13.877722    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:14.635541 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:14.635485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:15.412899 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:15.412866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:16.145287 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:16.145249    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:16.801705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:16.801675    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:16.862393 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:16.862358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:17.024495 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.024425    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:17.253794 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.253694    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:17.358753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.358695    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:17.437532 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.437490    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"]
Sep 17 03:43:17.441202 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.441183    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"
Sep 17 03:43:17.443222 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.443202    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-4a43bf\""
Sep 17 03:43:17.443321 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.443202    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-4a43bf-dockercfg-7crvr\""
Sep 17 03:43:17.447326 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.447301    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"]
Sep 17 03:43:17.542743 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.542650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db4ed815-862e-4f23-9288-57b8d7350ad7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j\" (UID: \"db4ed815-862e-4f23-9288-57b8d7350ad7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"
Sep 17 03:43:17.542743 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.542697    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87k8f\" (UniqueName: \"kubernetes.io/projected/db4ed815-862e-4f23-9288-57b8d7350ad7-kube-api-access-87k8f\") pod \"payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j\" (UID: \"db4ed815-862e-4f23-9288-57b8d7350ad7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"
Sep 17 03:43:17.643378 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.643326    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db4ed815-862e-4f23-9288-57b8d7350ad7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j\" (UID: \"db4ed815-862e-4f23-9288-57b8d7350ad7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"
Sep 17 03:43:17.643378 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.643388    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87k8f\" (UniqueName: \"kubernetes.io/projected/db4ed815-862e-4f23-9288-57b8d7350ad7-kube-api-access-87k8f\") pod \"payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j\" (UID: \"db4ed815-862e-4f23-9288-57b8d7350ad7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"
Sep 17 03:43:17.644071 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.644051    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db4ed815-862e-4f23-9288-57b8d7350ad7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j\" (UID: \"db4ed815-862e-4f23-9288-57b8d7350ad7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"
Sep 17 03:43:17.651019 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.650996    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87k8f\" (UniqueName: \"kubernetes.io/projected/db4ed815-862e-4f23-9288-57b8d7350ad7-kube-api-access-87k8f\") pod \"payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j\" (UID: \"db4ed815-862e-4f23-9288-57b8d7350ad7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"
Sep 17 03:43:17.752822 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:17.752776    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"
Sep 17 03:43:18.030124 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:18.030094    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"]
Sep 17 03:43:18.032755 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:43:18.032724    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb4ed815_862e_4f23_9288_57b8d7350ad7.slice/crio-9a369ba5a88397b38f65a4b56e26f6b5054bbf8051cff1c80a1d8ab0872fd42c WatchSource:0}: Error finding container 9a369ba5a88397b38f65a4b56e26f6b5054bbf8051cff1c80a1d8ab0872fd42c: Status 404 returned error can't find the container with id 9a369ba5a88397b38f65a4b56e26f6b5054bbf8051cff1c80a1d8ab0872fd42c
Sep 17 03:43:19.041573 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:19.041537    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j" event={"ID":"db4ed815-862e-4f23-9288-57b8d7350ad7","Type":"ContainerStarted","Data":"c2c23cad8e8d434a4a96fd10677b836faf3005bb8df1fd0207110a3842bafbc8"}
Sep 17 03:43:19.042016 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:19.041614    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"
Sep 17 03:43:19.042016 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:19.041640    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j" event={"ID":"db4ed815-862e-4f23-9288-57b8d7350ad7","Type":"ContainerStarted","Data":"9a369ba5a88397b38f65a4b56e26f6b5054bbf8051cff1c80a1d8ab0872fd42c"}
Sep 17 03:43:19.058810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:19.058756    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j" podStartSLOduration=2.058738539 podStartE2EDuration="2.058738539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:43:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:43:19.057263068 +0000 UTC m=+1427.401989147" watchObservedRunningTime="2026-09-17 03:43:19.058738539 +0000 UTC m=+1427.403464654"
Sep 17 03:43:20.995403 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:20.995371    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:43:30.048682 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:30.048587    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"
Sep 17 03:43:32.393388 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:32.393345    2576 scope.go:117] "RemoveContainer" containerID="c05ca5635984049437b5b2aa46a205af6180820ede80bfcca427cf2056a5f1e3"
Sep 17 03:43:38.539001 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:38.538962    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:43:38.835131 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:38.835094    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:39.563831 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:39.563801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:40.323086 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:40.323034    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:41.067644 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:41.067615    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:42.008146 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:42.008111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:42.105895 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:42.105860    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:42.551942 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:42.551912    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:42.882142 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:42.882061    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:43.030171 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:43.030119    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:43.157498 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:43.157386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:43.856396 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:43.856358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:44.716527 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:44.716490    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:45.665791 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:45.665734    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:45.789094 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:45.789058    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:45.882060 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:45.882025    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:46.414369 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:46.414325    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:47.323267 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:47.323236    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:48.007111 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:48.007049    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:48.791720 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:48.791668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:49.471219 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:49.471183    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:50.243694 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:50.243663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:50.941382 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:50.941340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:51.619231 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:51.619202    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:51.718920 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:51.718881    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:52.125243 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.125211    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:52.292307 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.292278    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:52.378943 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.378853    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"]
Sep 17 03:43:52.383903 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.383880    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"
Sep 17 03:43:52.386116 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.386094    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-26d5d1\""
Sep 17 03:43:52.386263 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.386156    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-26d5d1-dockercfg-clqf5\""
Sep 17 03:43:52.393970 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.393947    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"]
Sep 17 03:43:52.434617 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.434578    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"]
Sep 17 03:43:52.438032 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.438015    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"
Sep 17 03:43:52.447326 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.447299    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"]
Sep 17 03:43:52.478873 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.478833    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hxdn\" (UniqueName: \"kubernetes.io/projected/16377c3b-5ebb-44bf-904d-7ff996f9ed3b-kube-api-access-7hxdn\") pod \"payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb\" (UID: \"16377c3b-5ebb-44bf-904d-7ff996f9ed3b\") " pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"
Sep 17 03:43:52.479070 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.478877    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/015f3d4f-845e-41c8-8291-757d9658a967-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2\" (UID: \"015f3d4f-845e-41c8-8291-757d9658a967\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"
Sep 17 03:43:52.479146 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.479093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16377c3b-5ebb-44bf-904d-7ff996f9ed3b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb\" (UID: \"16377c3b-5ebb-44bf-904d-7ff996f9ed3b\") " pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"
Sep 17 03:43:52.479213 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.479145    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkplb\" (UniqueName: \"kubernetes.io/projected/015f3d4f-845e-41c8-8291-757d9658a967-kube-api-access-zkplb\") pod \"payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2\" (UID: \"015f3d4f-845e-41c8-8291-757d9658a967\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"
Sep 17 03:43:52.580057 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.580014    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16377c3b-5ebb-44bf-904d-7ff996f9ed3b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb\" (UID: \"16377c3b-5ebb-44bf-904d-7ff996f9ed3b\") " pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"
Sep 17 03:43:52.580057 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.580063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkplb\" (UniqueName: \"kubernetes.io/projected/015f3d4f-845e-41c8-8291-757d9658a967-kube-api-access-zkplb\") pod \"payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2\" (UID: \"015f3d4f-845e-41c8-8291-757d9658a967\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"
Sep 17 03:43:52.580517 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.580098    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hxdn\" (UniqueName: \"kubernetes.io/projected/16377c3b-5ebb-44bf-904d-7ff996f9ed3b-kube-api-access-7hxdn\") pod \"payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb\" (UID: \"16377c3b-5ebb-44bf-904d-7ff996f9ed3b\") " pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"
Sep 17 03:43:52.580517 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.580128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/015f3d4f-845e-41c8-8291-757d9658a967-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2\" (UID: \"015f3d4f-845e-41c8-8291-757d9658a967\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"
Sep 17 03:43:52.581061 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.581029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16377c3b-5ebb-44bf-904d-7ff996f9ed3b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb\" (UID: \"16377c3b-5ebb-44bf-904d-7ff996f9ed3b\") " pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"
Sep 17 03:43:52.581214 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.581182    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/015f3d4f-845e-41c8-8291-757d9658a967-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2\" (UID: \"015f3d4f-845e-41c8-8291-757d9658a967\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"
Sep 17 03:43:52.588272 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.588246    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hxdn\" (UniqueName: \"kubernetes.io/projected/16377c3b-5ebb-44bf-904d-7ff996f9ed3b-kube-api-access-7hxdn\") pod \"payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb\" (UID: \"16377c3b-5ebb-44bf-904d-7ff996f9ed3b\") " pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"
Sep 17 03:43:52.588405 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.588246    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkplb\" (UniqueName: \"kubernetes.io/projected/015f3d4f-845e-41c8-8291-757d9658a967-kube-api-access-zkplb\") pod \"payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2\" (UID: \"015f3d4f-845e-41c8-8291-757d9658a967\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"
Sep 17 03:43:52.630506 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.630395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:43:52.694794 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.694761    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"
Sep 17 03:43:52.749862 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.749834    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"
Sep 17 03:43:52.966302 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:52.966276    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"]
Sep 17 03:43:52.967888 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:43:52.967859    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod015f3d4f_845e_41c8_8291_757d9658a967.slice/crio-1f5ab00870bf09f92447e334096e0bd5bda34b420463b2b25106d163f7969b50 WatchSource:0}: Error finding container 1f5ab00870bf09f92447e334096e0bd5bda34b420463b2b25106d163f7969b50: Status 404 returned error can't find the container with id 1f5ab00870bf09f92447e334096e0bd5bda34b420463b2b25106d163f7969b50
Sep 17 03:43:53.031852 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:53.031826    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"]
Sep 17 03:43:53.032713 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:43:53.032693    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16377c3b_5ebb_44bf_904d_7ff996f9ed3b.slice/crio-199c2481d7aaeb18e07310e5f20273a6d75ac670e8cef7c307c954a762edd8fa WatchSource:0}: Error finding container 199c2481d7aaeb18e07310e5f20273a6d75ac670e8cef7c307c954a762edd8fa: Status 404 returned error can't find the container with id 199c2481d7aaeb18e07310e5f20273a6d75ac670e8cef7c307c954a762edd8fa
Sep 17 03:43:53.249794 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:53.249692    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2" event={"ID":"015f3d4f-845e-41c8-8291-757d9658a967","Type":"ContainerStarted","Data":"3eac3a922f7b8a9fd76ee869fe5484d78708a60b3311452cfdfec421ac2e886b"}
Sep 17 03:43:53.249794 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:53.249753    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2" event={"ID":"015f3d4f-845e-41c8-8291-757d9658a967","Type":"ContainerStarted","Data":"1f5ab00870bf09f92447e334096e0bd5bda34b420463b2b25106d163f7969b50"}
Sep 17 03:43:53.250046 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:53.249898    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"
Sep 17 03:43:53.251547 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:53.251521    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb" event={"ID":"16377c3b-5ebb-44bf-904d-7ff996f9ed3b","Type":"ContainerStarted","Data":"df482e786fec80241d9d6570f2384e94a85b493ad6d0f7f73838db60e546a5d6"}
Sep 17 03:43:53.251661 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:53.251560    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb" event={"ID":"16377c3b-5ebb-44bf-904d-7ff996f9ed3b","Type":"ContainerStarted","Data":"199c2481d7aaeb18e07310e5f20273a6d75ac670e8cef7c307c954a762edd8fa"}
Sep 17 03:43:53.251661 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:53.251634    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"
Sep 17 03:43:53.266747 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:53.266696    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2" podStartSLOduration=1.266683552 podStartE2EDuration="1.266683552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:43:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:43:53.263900105 +0000 UTC m=+1461.608626183" watchObservedRunningTime="2026-09-17 03:43:53.266683552 +0000 UTC m=+1461.611409630"
Sep 17 03:43:53.283155 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:53.283093    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb" podStartSLOduration=1.283073192 podStartE2EDuration="1.283073192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:43:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:43:53.278409188 +0000 UTC m=+1461.623135267" watchObservedRunningTime="2026-09-17 03:43:53.283073192 +0000 UTC m=+1461.627799272"
Sep 17 03:43:56.130361 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:43:56.130324    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:01.945624 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:01.945582    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:02.397597 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:02.397524    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:03.390812 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:03.390777    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:04.044016 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:04.043981    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:04.262926 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:04.262893    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"
Sep 17 03:44:04.263125 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:04.263006    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"
Sep 17 03:44:04.718707 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:04.718655    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:05.501035 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:05.501001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:06.229723 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:06.229687    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:07.081873 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:07.081838    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:07.151133 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:07.151105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:07.909977 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:07.909943    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:08.614300 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:08.614264    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:09.428570 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:09.428537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:10.215167 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:10.215139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:10.677672 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:10.677639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:10.910445 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:10.910399    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:11.588795 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:11.588763    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:12.332114 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:12.332074    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:12.979116 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:12.979074    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:13.810336 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:13.810304    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:14.624167 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:14.624131    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:15.301665 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:15.301620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:16.110365 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:16.110327    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:16.138584 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:16.138547    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:44:16.798233 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:16.798195    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:17.598273 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:17.598239    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:18.330940 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:18.330903    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:19.109759 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:19.109729    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:19.758824 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:19.758791    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:20.886593 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:20.886553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:20.917220 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:20.917188    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"]
Sep 17 03:44:20.917462 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:20.917414    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9" podUID="35aa00e6-c101-4854-9a2a-cad7050f8fd5" containerName="payload-processing" containerID="cri-o://cf3529f161676277bbc48c8569aace1ceeaa8f2237c0e137cf791f665df6d95a" gracePeriod=30
Sep 17 03:44:20.925004 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:20.924973    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"]
Sep 17 03:44:20.925221 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:20.925198    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq" podUID="e45dbaad-e524-4b5a-ae25-d449de969fe2" containerName="payload-pre-processing" containerID="cri-o://3a2aaace0cb450bf99a9561d25638495944eaf1e38f8dd2a4fba17fd30c66363" gracePeriod=30
Sep 17 03:44:20.954115 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:20.954087    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:21.216700 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.216672    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"
Sep 17 03:44:21.219935 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.219911    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"
Sep 17 03:44:21.360773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.360727    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5rnc\" (UniqueName: \"kubernetes.io/projected/35aa00e6-c101-4854-9a2a-cad7050f8fd5-kube-api-access-f5rnc\") pod \"35aa00e6-c101-4854-9a2a-cad7050f8fd5\" (UID: \"35aa00e6-c101-4854-9a2a-cad7050f8fd5\") "
Sep 17 03:44:21.360961 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.360805    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35aa00e6-c101-4854-9a2a-cad7050f8fd5-plugins-config-volume\") pod \"35aa00e6-c101-4854-9a2a-cad7050f8fd5\" (UID: \"35aa00e6-c101-4854-9a2a-cad7050f8fd5\") "
Sep 17 03:44:21.360961 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.360918    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmf64\" (UniqueName: \"kubernetes.io/projected/e45dbaad-e524-4b5a-ae25-d449de969fe2-kube-api-access-jmf64\") pod \"e45dbaad-e524-4b5a-ae25-d449de969fe2\" (UID: \"e45dbaad-e524-4b5a-ae25-d449de969fe2\") "
Sep 17 03:44:21.360961 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.360954    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e45dbaad-e524-4b5a-ae25-d449de969fe2-plugins-config-volume\") pod \"e45dbaad-e524-4b5a-ae25-d449de969fe2\" (UID: \"e45dbaad-e524-4b5a-ae25-d449de969fe2\") "
Sep 17 03:44:21.361355 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.361321    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35aa00e6-c101-4854-9a2a-cad7050f8fd5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "35aa00e6-c101-4854-9a2a-cad7050f8fd5" (UID: "35aa00e6-c101-4854-9a2a-cad7050f8fd5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:44:21.361465 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.361369    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e45dbaad-e524-4b5a-ae25-d449de969fe2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e45dbaad-e524-4b5a-ae25-d449de969fe2" (UID: "e45dbaad-e524-4b5a-ae25-d449de969fe2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:44:21.363055 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.363030    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45dbaad-e524-4b5a-ae25-d449de969fe2-kube-api-access-jmf64" (OuterVolumeSpecName: "kube-api-access-jmf64") pod "e45dbaad-e524-4b5a-ae25-d449de969fe2" (UID: "e45dbaad-e524-4b5a-ae25-d449de969fe2"). InnerVolumeSpecName "kube-api-access-jmf64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:44:21.363144 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.363117    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35aa00e6-c101-4854-9a2a-cad7050f8fd5-kube-api-access-f5rnc" (OuterVolumeSpecName: "kube-api-access-f5rnc") pod "35aa00e6-c101-4854-9a2a-cad7050f8fd5" (UID: "35aa00e6-c101-4854-9a2a-cad7050f8fd5"). InnerVolumeSpecName "kube-api-access-f5rnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:44:21.428496 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.428459    2576 generic.go:356] "Generic (PLEG): container finished" podID="e45dbaad-e524-4b5a-ae25-d449de969fe2" containerID="3a2aaace0cb450bf99a9561d25638495944eaf1e38f8dd2a4fba17fd30c66363" exitCode=0
Sep 17 03:44:21.428694 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.428533    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"
Sep 17 03:44:21.428694 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.428555    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq" event={"ID":"e45dbaad-e524-4b5a-ae25-d449de969fe2","Type":"ContainerDied","Data":"3a2aaace0cb450bf99a9561d25638495944eaf1e38f8dd2a4fba17fd30c66363"}
Sep 17 03:44:21.428694 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.428612    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq" event={"ID":"e45dbaad-e524-4b5a-ae25-d449de969fe2","Type":"ContainerDied","Data":"a16a160968d4c715ac67276d1d940377402c601339bce34cea717958f8d2325c"}
Sep 17 03:44:21.428694 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.428633    2576 scope.go:117] "RemoveContainer" containerID="3a2aaace0cb450bf99a9561d25638495944eaf1e38f8dd2a4fba17fd30c66363"
Sep 17 03:44:21.430130 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.430110    2576 generic.go:356] "Generic (PLEG): container finished" podID="35aa00e6-c101-4854-9a2a-cad7050f8fd5" containerID="cf3529f161676277bbc48c8569aace1ceeaa8f2237c0e137cf791f665df6d95a" exitCode=0
Sep 17 03:44:21.430243 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.430148    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9" event={"ID":"35aa00e6-c101-4854-9a2a-cad7050f8fd5","Type":"ContainerDied","Data":"cf3529f161676277bbc48c8569aace1ceeaa8f2237c0e137cf791f665df6d95a"}
Sep 17 03:44:21.430243 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.430177    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"
Sep 17 03:44:21.430243 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.430183    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9" event={"ID":"35aa00e6-c101-4854-9a2a-cad7050f8fd5","Type":"ContainerDied","Data":"8f623ea76cd0367efbeee68837aed7dff9d537120c0f7f8929920d13e3731c42"}
Sep 17 03:44:21.439858 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.439839    2576 scope.go:117] "RemoveContainer" containerID="3a2aaace0cb450bf99a9561d25638495944eaf1e38f8dd2a4fba17fd30c66363"
Sep 17 03:44:21.440224 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:44:21.440201    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a2aaace0cb450bf99a9561d25638495944eaf1e38f8dd2a4fba17fd30c66363\": container with ID starting with 3a2aaace0cb450bf99a9561d25638495944eaf1e38f8dd2a4fba17fd30c66363 not found: ID does not exist" containerID="3a2aaace0cb450bf99a9561d25638495944eaf1e38f8dd2a4fba17fd30c66363"
Sep 17 03:44:21.440325 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.440236    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a2aaace0cb450bf99a9561d25638495944eaf1e38f8dd2a4fba17fd30c66363"} err="failed to get container status \"3a2aaace0cb450bf99a9561d25638495944eaf1e38f8dd2a4fba17fd30c66363\": rpc error: code = NotFound desc = could not find container \"3a2aaace0cb450bf99a9561d25638495944eaf1e38f8dd2a4fba17fd30c66363\": container with ID starting with 3a2aaace0cb450bf99a9561d25638495944eaf1e38f8dd2a4fba17fd30c66363 not found: ID does not exist"
Sep 17 03:44:21.440325 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.440261    2576 scope.go:117] "RemoveContainer" containerID="cf3529f161676277bbc48c8569aace1ceeaa8f2237c0e137cf791f665df6d95a"
Sep 17 03:44:21.449331 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.449310    2576 scope.go:117] "RemoveContainer" containerID="cf3529f161676277bbc48c8569aace1ceeaa8f2237c0e137cf791f665df6d95a"
Sep 17 03:44:21.449722 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:44:21.449700    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf3529f161676277bbc48c8569aace1ceeaa8f2237c0e137cf791f665df6d95a\": container with ID starting with cf3529f161676277bbc48c8569aace1ceeaa8f2237c0e137cf791f665df6d95a not found: ID does not exist" containerID="cf3529f161676277bbc48c8569aace1ceeaa8f2237c0e137cf791f665df6d95a"
Sep 17 03:44:21.449806 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.449729    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf3529f161676277bbc48c8569aace1ceeaa8f2237c0e137cf791f665df6d95a"} err="failed to get container status \"cf3529f161676277bbc48c8569aace1ceeaa8f2237c0e137cf791f665df6d95a\": rpc error: code = NotFound desc = could not find container \"cf3529f161676277bbc48c8569aace1ceeaa8f2237c0e137cf791f665df6d95a\": container with ID starting with cf3529f161676277bbc48c8569aace1ceeaa8f2237c0e137cf791f665df6d95a not found: ID does not exist"
Sep 17 03:44:21.452311 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.452293    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"]
Sep 17 03:44:21.458838 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.458810    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1afd34-59465b99db-dv2rq"]
Sep 17 03:44:21.462007 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.461980    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmf64\" (UniqueName: \"kubernetes.io/projected/e45dbaad-e524-4b5a-ae25-d449de969fe2-kube-api-access-jmf64\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:44:21.462135 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.462008    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e45dbaad-e524-4b5a-ae25-d449de969fe2-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:44:21.462135 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.462024    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5rnc\" (UniqueName: \"kubernetes.io/projected/35aa00e6-c101-4854-9a2a-cad7050f8fd5-kube-api-access-f5rnc\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:44:21.462135 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.462037    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35aa00e6-c101-4854-9a2a-cad7050f8fd5-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:44:21.468142 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.468116    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"]
Sep 17 03:44:21.471803 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:21.471780    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1afd34-67b6b64cc7-7m6f9"]
Sep 17 03:44:22.178810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:22.178778    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:22.284969 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:22.284937    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:22.299792 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:22.299723    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35aa00e6-c101-4854-9a2a-cad7050f8fd5" path="/var/lib/kubelet/pods/35aa00e6-c101-4854-9a2a-cad7050f8fd5/volumes"
Sep 17 03:44:22.300152 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:22.300133    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e45dbaad-e524-4b5a-ae25-d449de969fe2" path="/var/lib/kubelet/pods/e45dbaad-e524-4b5a-ae25-d449de969fe2/volumes"
Sep 17 03:44:22.743108 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:22.743078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:23.514800 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:23.514765    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:24.326271 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:24.326239    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:24.952970 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:24.952938    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:25.594549 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:25.594485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:26.525448 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:26.525404    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:27.466390 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:27.466357    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:27.540952 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:27.540918    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:27.742454 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:27.742361    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:32.196540 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.196502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:32.230907 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.230871    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"]
Sep 17 03:44:32.231158 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.231125    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j" podUID="db4ed815-862e-4f23-9288-57b8d7350ad7" containerName="payload-pre-processing" containerID="cri-o://c2c23cad8e8d434a4a96fd10677b836faf3005bb8df1fd0207110a3842bafbc8" gracePeriod=30
Sep 17 03:44:32.295883 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.295835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:32.313499 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.313467    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:44:32.313846 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.313831    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:44:32.386709 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.386681    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:32.416529 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.416504    2576 scope.go:117] "RemoveContainer" containerID="fbc0e8bb225b325ee3feedb90e73e8c9adb8f67ac180fb786d6231892b3d9567"
Sep 17 03:44:32.439970 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.439936    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:32.500633 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.500598    2576 generic.go:356] "Generic (PLEG): container finished" podID="db4ed815-862e-4f23-9288-57b8d7350ad7" containerID="c2c23cad8e8d434a4a96fd10677b836faf3005bb8df1fd0207110a3842bafbc8" exitCode=0
Sep 17 03:44:32.500780 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.500637    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j" event={"ID":"db4ed815-862e-4f23-9288-57b8d7350ad7","Type":"ContainerDied","Data":"c2c23cad8e8d434a4a96fd10677b836faf3005bb8df1fd0207110a3842bafbc8"}
Sep 17 03:44:32.500780 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.500702    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j" event={"ID":"db4ed815-862e-4f23-9288-57b8d7350ad7","Type":"ContainerDied","Data":"9a369ba5a88397b38f65a4b56e26f6b5054bbf8051cff1c80a1d8ab0872fd42c"}
Sep 17 03:44:32.500780 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.500729    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a369ba5a88397b38f65a4b56e26f6b5054bbf8051cff1c80a1d8ab0872fd42c"
Sep 17 03:44:32.501012 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.500998    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"
Sep 17 03:44:32.560203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.560071    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87k8f\" (UniqueName: \"kubernetes.io/projected/db4ed815-862e-4f23-9288-57b8d7350ad7-kube-api-access-87k8f\") pod \"db4ed815-862e-4f23-9288-57b8d7350ad7\" (UID: \"db4ed815-862e-4f23-9288-57b8d7350ad7\") "
Sep 17 03:44:32.560203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.560149    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db4ed815-862e-4f23-9288-57b8d7350ad7-plugins-config-volume\") pod \"db4ed815-862e-4f23-9288-57b8d7350ad7\" (UID: \"db4ed815-862e-4f23-9288-57b8d7350ad7\") "
Sep 17 03:44:32.560662 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.560634    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db4ed815-862e-4f23-9288-57b8d7350ad7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "db4ed815-862e-4f23-9288-57b8d7350ad7" (UID: "db4ed815-862e-4f23-9288-57b8d7350ad7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:44:32.562378 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.562353    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db4ed815-862e-4f23-9288-57b8d7350ad7-kube-api-access-87k8f" (OuterVolumeSpecName: "kube-api-access-87k8f") pod "db4ed815-862e-4f23-9288-57b8d7350ad7" (UID: "db4ed815-862e-4f23-9288-57b8d7350ad7"). InnerVolumeSpecName "kube-api-access-87k8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:44:32.662027 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.661997    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87k8f\" (UniqueName: \"kubernetes.io/projected/db4ed815-862e-4f23-9288-57b8d7350ad7-kube-api-access-87k8f\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:44:32.662027 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.662022    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db4ed815-862e-4f23-9288-57b8d7350ad7-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:44:32.734128 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.734040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:32.905899 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:32.905862    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:33.506885 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:33.506853    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"
Sep 17 03:44:33.530658 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:33.530626    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"]
Sep 17 03:44:33.537325 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:33.537293    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4a43bf-69c5449578-d642j"]
Sep 17 03:44:33.553877 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:33.553849    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:34.295487 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:34.295451    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db4ed815-862e-4f23-9288-57b8d7350ad7" path="/var/lib/kubelet/pods/db4ed815-862e-4f23-9288-57b8d7350ad7/volumes"
Sep 17 03:44:34.598058 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:34.597989    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:35.227840 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:35.227801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:36.004989 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:36.004952    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:36.810318 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:36.810283    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:37.569076 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:37.569039    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:37.670350 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:37.670324    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:38.417990 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:38.417955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:38.480445 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:38.480407    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:38.656136 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:38.656108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:39.190262 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:39.190228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:39.900315 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:39.900289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:40.676821 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:40.676778    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:41.298937 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:41.298907    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:41.943137 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:41.943106    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:42.698328 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:42.698294    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:43.343749 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:43.343719    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:44.723806 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:44.723775    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:45.072041 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:45.072004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:51.031274 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:51.031238    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:44:54.437179 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.437100    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:54.465214 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.465183    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"]
Sep 17 03:44:54.465473 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.465444    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb" podUID="16377c3b-5ebb-44bf-904d-7ff996f9ed3b" containerName="payload-processing" containerID="cri-o://df482e786fec80241d9d6570f2384e94a85b493ad6d0f7f73838db60e546a5d6" gracePeriod=30
Sep 17 03:44:54.473077 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.473053    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"]
Sep 17 03:44:54.473299 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.473273    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2" podUID="015f3d4f-845e-41c8-8291-757d9658a967" containerName="payload-pre-processing" containerID="cri-o://3eac3a922f7b8a9fd76ee869fe5484d78708a60b3311452cfdfec421ac2e886b" gracePeriod=30
Sep 17 03:44:54.506978 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.506798    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:54.609694 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.609663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:54.631154 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.631124    2576 generic.go:356] "Generic (PLEG): container finished" podID="16377c3b-5ebb-44bf-904d-7ff996f9ed3b" containerID="df482e786fec80241d9d6570f2384e94a85b493ad6d0f7f73838db60e546a5d6" exitCode=0
Sep 17 03:44:54.631333 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.631197    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb" event={"ID":"16377c3b-5ebb-44bf-904d-7ff996f9ed3b","Type":"ContainerDied","Data":"df482e786fec80241d9d6570f2384e94a85b493ad6d0f7f73838db60e546a5d6"}
Sep 17 03:44:54.633234 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.633197    2576 generic.go:356] "Generic (PLEG): container finished" podID="015f3d4f-845e-41c8-8291-757d9658a967" containerID="3eac3a922f7b8a9fd76ee869fe5484d78708a60b3311452cfdfec421ac2e886b" exitCode=0
Sep 17 03:44:54.633419 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.633287    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2" event={"ID":"015f3d4f-845e-41c8-8291-757d9658a967","Type":"ContainerDied","Data":"3eac3a922f7b8a9fd76ee869fe5484d78708a60b3311452cfdfec421ac2e886b"}
Sep 17 03:44:54.695819 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.695745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:54.759991 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.759956    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"
Sep 17 03:44:54.762774 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.762755    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"
Sep 17 03:44:54.860602 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.860568    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16377c3b-5ebb-44bf-904d-7ff996f9ed3b-plugins-config-volume\") pod \"16377c3b-5ebb-44bf-904d-7ff996f9ed3b\" (UID: \"16377c3b-5ebb-44bf-904d-7ff996f9ed3b\") "
Sep 17 03:44:54.860801 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.860621    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkplb\" (UniqueName: \"kubernetes.io/projected/015f3d4f-845e-41c8-8291-757d9658a967-kube-api-access-zkplb\") pod \"015f3d4f-845e-41c8-8291-757d9658a967\" (UID: \"015f3d4f-845e-41c8-8291-757d9658a967\") "
Sep 17 03:44:54.860801 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.860649    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/015f3d4f-845e-41c8-8291-757d9658a967-plugins-config-volume\") pod \"015f3d4f-845e-41c8-8291-757d9658a967\" (UID: \"015f3d4f-845e-41c8-8291-757d9658a967\") "
Sep 17 03:44:54.860801 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.860763    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hxdn\" (UniqueName: \"kubernetes.io/projected/16377c3b-5ebb-44bf-904d-7ff996f9ed3b-kube-api-access-7hxdn\") pod \"16377c3b-5ebb-44bf-904d-7ff996f9ed3b\" (UID: \"16377c3b-5ebb-44bf-904d-7ff996f9ed3b\") "
Sep 17 03:44:54.861004 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.860986    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/015f3d4f-845e-41c8-8291-757d9658a967-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "015f3d4f-845e-41c8-8291-757d9658a967" (UID: "015f3d4f-845e-41c8-8291-757d9658a967"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:44:54.861064 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.861019    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16377c3b-5ebb-44bf-904d-7ff996f9ed3b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "16377c3b-5ebb-44bf-904d-7ff996f9ed3b" (UID: "16377c3b-5ebb-44bf-904d-7ff996f9ed3b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:44:54.861153 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.861135    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/015f3d4f-845e-41c8-8291-757d9658a967-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:44:54.861201 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.861161    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16377c3b-5ebb-44bf-904d-7ff996f9ed3b-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:44:54.862854 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.862829    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16377c3b-5ebb-44bf-904d-7ff996f9ed3b-kube-api-access-7hxdn" (OuterVolumeSpecName: "kube-api-access-7hxdn") pod "16377c3b-5ebb-44bf-904d-7ff996f9ed3b" (UID: "16377c3b-5ebb-44bf-904d-7ff996f9ed3b"). InnerVolumeSpecName "kube-api-access-7hxdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:44:54.862914 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.862854    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/015f3d4f-845e-41c8-8291-757d9658a967-kube-api-access-zkplb" (OuterVolumeSpecName: "kube-api-access-zkplb") pod "015f3d4f-845e-41c8-8291-757d9658a967" (UID: "015f3d4f-845e-41c8-8291-757d9658a967"). InnerVolumeSpecName "kube-api-access-zkplb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:44:54.962282 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.962182    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hxdn\" (UniqueName: \"kubernetes.io/projected/16377c3b-5ebb-44bf-904d-7ff996f9ed3b-kube-api-access-7hxdn\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:44:54.962282 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.962214    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkplb\" (UniqueName: \"kubernetes.io/projected/015f3d4f-845e-41c8-8291-757d9658a967-kube-api-access-zkplb\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:44:54.974196 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:54.974161    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:55.074357 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:55.074317    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:55.640174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:55.640141    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"
Sep 17 03:44:55.640174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:55.640153    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb" event={"ID":"16377c3b-5ebb-44bf-904d-7ff996f9ed3b","Type":"ContainerDied","Data":"199c2481d7aaeb18e07310e5f20273a6d75ac670e8cef7c307c954a762edd8fa"}
Sep 17 03:44:55.640723 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:55.640232    2576 scope.go:117] "RemoveContainer" containerID="df482e786fec80241d9d6570f2384e94a85b493ad6d0f7f73838db60e546a5d6"
Sep 17 03:44:55.641695 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:55.641675    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2" event={"ID":"015f3d4f-845e-41c8-8291-757d9658a967","Type":"ContainerDied","Data":"1f5ab00870bf09f92447e334096e0bd5bda34b420463b2b25106d163f7969b50"}
Sep 17 03:44:55.641778 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:55.641708    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"
Sep 17 03:44:55.651832 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:55.651810    2576 scope.go:117] "RemoveContainer" containerID="3eac3a922f7b8a9fd76ee869fe5484d78708a60b3311452cfdfec421ac2e886b"
Sep 17 03:44:55.666163 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:55.666133    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"]
Sep 17 03:44:55.669960 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:55.669936    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-26d5d1-658dc8f845-qndxb"]
Sep 17 03:44:55.679250 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:55.679210    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"]
Sep 17 03:44:55.685611 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:55.685581    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-26d5d1-76b9749cdb-cdpx2"]
Sep 17 03:44:55.823507 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:55.823476    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:56.295133 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:56.295095    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="015f3d4f-845e-41c8-8291-757d9658a967" path="/var/lib/kubelet/pods/015f3d4f-845e-41c8-8291-757d9658a967/volumes"
Sep 17 03:44:56.295490 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:56.295476    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16377c3b-5ebb-44bf-904d-7ff996f9ed3b" path="/var/lib/kubelet/pods/16377c3b-5ebb-44bf-904d-7ff996f9ed3b/volumes"
Sep 17 03:44:56.503890 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:56.503862    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:57.278739 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:57.278709    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:58.009939 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:58.009898    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:58.253604 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:58.253568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:58.898989 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:58.898956    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:44:59.714077 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:44:59.714044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:00.137296 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137257    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-57958f-29826945-gmp4k"]
Sep 17 03:45:00.137725 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137662    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16377c3b-5ebb-44bf-904d-7ff996f9ed3b" containerName="payload-processing"
Sep 17 03:45:00.137725 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137674    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="16377c3b-5ebb-44bf-904d-7ff996f9ed3b" containerName="payload-processing"
Sep 17 03:45:00.137725 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137682    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db4ed815-862e-4f23-9288-57b8d7350ad7" containerName="payload-pre-processing"
Sep 17 03:45:00.137725 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137687    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="db4ed815-862e-4f23-9288-57b8d7350ad7" containerName="payload-pre-processing"
Sep 17 03:45:00.137725 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137697    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35aa00e6-c101-4854-9a2a-cad7050f8fd5" containerName="payload-processing"
Sep 17 03:45:00.137725 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137703    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="35aa00e6-c101-4854-9a2a-cad7050f8fd5" containerName="payload-processing"
Sep 17 03:45:00.137725 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137718    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e45dbaad-e524-4b5a-ae25-d449de969fe2" containerName="payload-pre-processing"
Sep 17 03:45:00.137725 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137723    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e45dbaad-e524-4b5a-ae25-d449de969fe2" containerName="payload-pre-processing"
Sep 17 03:45:00.137973 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137731    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="015f3d4f-845e-41c8-8291-757d9658a967" containerName="payload-pre-processing"
Sep 17 03:45:00.137973 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137737    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="015f3d4f-845e-41c8-8291-757d9658a967" containerName="payload-pre-processing"
Sep 17 03:45:00.137973 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137800    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="015f3d4f-845e-41c8-8291-757d9658a967" containerName="payload-pre-processing"
Sep 17 03:45:00.137973 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137810    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="db4ed815-862e-4f23-9288-57b8d7350ad7" containerName="payload-pre-processing"
Sep 17 03:45:00.137973 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137816    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="35aa00e6-c101-4854-9a2a-cad7050f8fd5" containerName="payload-processing"
Sep 17 03:45:00.137973 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137821    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e45dbaad-e524-4b5a-ae25-d449de969fe2" containerName="payload-pre-processing"
Sep 17 03:45:00.137973 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.137828    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="16377c3b-5ebb-44bf-904d-7ff996f9ed3b" containerName="payload-processing"
Sep 17 03:45:00.142639 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.142608    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-57958f-29826945-gmp4k"
Sep 17 03:45:00.145174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.145153    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8vwnw\""
Sep 17 03:45:00.153686 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.153660    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-57958f-29826945-gmp4k"]
Sep 17 03:45:00.155581 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.155560    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-57958f-29826945-gmp4k"
Sep 17 03:45:00.434613 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.434582    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-57958f-29826945-gmp4k"]
Sep 17 03:45:00.436579 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:45:00.436550    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb75164ca_32a6_4aa0_9b71_5cba18cde94b.slice/crio-fd3131bfba5751b29845daa430d30549cd16c58503309abe3953dc38bc16712a WatchSource:0}: Error finding container fd3131bfba5751b29845daa430d30549cd16c58503309abe3953dc38bc16712a: Status 404 returned error can't find the container with id fd3131bfba5751b29845daa430d30549cd16c58503309abe3953dc38bc16712a
Sep 17 03:45:00.455333 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.455307    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:00.674957 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:00.674922    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-57958f-29826945-gmp4k" event={"ID":"b75164ca-32a6-4aa0-9b71-5cba18cde94b","Type":"ContainerStarted","Data":"fd3131bfba5751b29845daa430d30549cd16c58503309abe3953dc38bc16712a"}
Sep 17 03:45:01.138258 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:01.138229    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:01.484382 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:01.484352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:01.592042 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:01.592007    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:01.685843 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:01.685743    2576 generic.go:356] "Generic (PLEG): container finished" podID="b75164ca-32a6-4aa0-9b71-5cba18cde94b" containerID="1300bce36fd8f7731b93f7c8a0a2f3e7365654ded1b0713cdb124a5c269083dd" exitCode=0
Sep 17 03:45:01.686001 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:01.685916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-57958f-29826945-gmp4k" event={"ID":"b75164ca-32a6-4aa0-9b71-5cba18cde94b","Type":"ContainerDied","Data":"1300bce36fd8f7731b93f7c8a0a2f3e7365654ded1b0713cdb124a5c269083dd"}
Sep 17 03:45:01.729020 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:01.728992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:02.111203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:02.111152    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:02.841765 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:02.841740    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-57958f-29826945-gmp4k"
Sep 17 03:45:02.925078 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:02.925040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:03.186766 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:03.186718    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"]
Sep 17 03:45:03.189784 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:03.189745    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"]
Sep 17 03:45:03.280744 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:03.280715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:03.702358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:03.702330    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-57958f-29826945-gmp4k"
Sep 17 03:45:03.702358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:03.702343    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-57958f-29826945-gmp4k" event={"ID":"b75164ca-32a6-4aa0-9b71-5cba18cde94b","Type":"ContainerDied","Data":"fd3131bfba5751b29845daa430d30549cd16c58503309abe3953dc38bc16712a"}
Sep 17 03:45:03.702583 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:03.702385    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd3131bfba5751b29845daa430d30549cd16c58503309abe3953dc38bc16712a"
Sep 17 03:45:03.720448 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:03.720410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:04.606636 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:04.606604    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:05.413673 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:05.413639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:06.093092 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:06.093059    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:06.890652 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:06.890618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:07.527072 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:07.527040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:08.595974 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:08.595937    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:09.197346 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:09.197312    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:09.298494 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:09.298455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:12.182128 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:12.182095    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:12.697509 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:12.697462    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"]
Sep 17 03:45:12.697767 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:12.697742    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22" podUID="c75c2f50-4470-4207-b6d2-ad67461396a4" containerName="maas-api" containerID="cri-o://4485f7ec3044dc75ca4972701c6f9a81cfd5e31ea3c85a732b9502aed455b2a9" gracePeriod=30
Sep 17 03:45:12.725291 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:12.725221    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-57958f-29826945-gmp4k"]
Sep 17 03:45:12.726743 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:12.726708    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:12.727958 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:12.727939    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-57958f-29826945-gmp4k"]
Sep 17 03:45:12.792480 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:12.792420    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:12.860716 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:12.860686    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:12.922196 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:12.922140    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:12.957312 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:12.957245    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:45:12.988091 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:12.987984    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:13.029083 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.029044    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c75c2f50-4470-4207-b6d2-ad67461396a4-maas-api-tls\") pod \"c75c2f50-4470-4207-b6d2-ad67461396a4\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") "
Sep 17 03:45:13.029083 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.029096    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c75c2f50-4470-4207-b6d2-ad67461396a4-openshift-service-ca\") pod \"c75c2f50-4470-4207-b6d2-ad67461396a4\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") "
Sep 17 03:45:13.029490 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.029123    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sf2w\" (UniqueName: \"kubernetes.io/projected/c75c2f50-4470-4207-b6d2-ad67461396a4-kube-api-access-2sf2w\") pod \"c75c2f50-4470-4207-b6d2-ad67461396a4\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") "
Sep 17 03:45:13.029559 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.029544    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c75c2f50-4470-4207-b6d2-ad67461396a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c75c2f50-4470-4207-b6d2-ad67461396a4" (UID: "c75c2f50-4470-4207-b6d2-ad67461396a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:45:13.031472 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.031421    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c75c2f50-4470-4207-b6d2-ad67461396a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c75c2f50-4470-4207-b6d2-ad67461396a4" (UID: "c75c2f50-4470-4207-b6d2-ad67461396a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:45:13.031472 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.031449    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c75c2f50-4470-4207-b6d2-ad67461396a4-kube-api-access-2sf2w" (OuterVolumeSpecName: "kube-api-access-2sf2w") pod "c75c2f50-4470-4207-b6d2-ad67461396a4" (UID: "c75c2f50-4470-4207-b6d2-ad67461396a4"). InnerVolumeSpecName "kube-api-access-2sf2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:45:13.130710 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.130678    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c75c2f50-4470-4207-b6d2-ad67461396a4-metrics-certs\") pod \"c75c2f50-4470-4207-b6d2-ad67461396a4\" (UID: \"c75c2f50-4470-4207-b6d2-ad67461396a4\") "
Sep 17 03:45:13.130964 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.130948    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c75c2f50-4470-4207-b6d2-ad67461396a4-maas-api-tls\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:13.131027 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.130970    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c75c2f50-4470-4207-b6d2-ad67461396a4-openshift-service-ca\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:13.131027 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.130985    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sf2w\" (UniqueName: \"kubernetes.io/projected/c75c2f50-4470-4207-b6d2-ad67461396a4-kube-api-access-2sf2w\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:13.132755 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.132733    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c75c2f50-4470-4207-b6d2-ad67461396a4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c75c2f50-4470-4207-b6d2-ad67461396a4" (UID: "c75c2f50-4470-4207-b6d2-ad67461396a4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:45:13.232450 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.232347    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c75c2f50-4470-4207-b6d2-ad67461396a4-metrics-certs\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:13.263202 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.263172    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:13.363740 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.363704    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:13.783187 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.783151    2576 generic.go:356] "Generic (PLEG): container finished" podID="c75c2f50-4470-4207-b6d2-ad67461396a4" containerID="4485f7ec3044dc75ca4972701c6f9a81cfd5e31ea3c85a732b9502aed455b2a9" exitCode=0
Sep 17 03:45:13.783392 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.783207    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"
Sep 17 03:45:13.783392 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.783207    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22" event={"ID":"c75c2f50-4470-4207-b6d2-ad67461396a4","Type":"ContainerDied","Data":"4485f7ec3044dc75ca4972701c6f9a81cfd5e31ea3c85a732b9502aed455b2a9"}
Sep 17 03:45:13.783392 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.783347    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22" event={"ID":"c75c2f50-4470-4207-b6d2-ad67461396a4","Type":"ContainerDied","Data":"0d1bc3152a277aacd51e1f6d27a7b5473132de37ee9da30e4fff191f85934950"}
Sep 17 03:45:13.783392 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.783388    2576 scope.go:117] "RemoveContainer" containerID="4485f7ec3044dc75ca4972701c6f9a81cfd5e31ea3c85a732b9502aed455b2a9"
Sep 17 03:45:13.793155 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.793137    2576 scope.go:117] "RemoveContainer" containerID="4485f7ec3044dc75ca4972701c6f9a81cfd5e31ea3c85a732b9502aed455b2a9"
Sep 17 03:45:13.793476 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:45:13.793455    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4485f7ec3044dc75ca4972701c6f9a81cfd5e31ea3c85a732b9502aed455b2a9\": container with ID starting with 4485f7ec3044dc75ca4972701c6f9a81cfd5e31ea3c85a732b9502aed455b2a9 not found: ID does not exist" containerID="4485f7ec3044dc75ca4972701c6f9a81cfd5e31ea3c85a732b9502aed455b2a9"
Sep 17 03:45:13.793537 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.793485    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4485f7ec3044dc75ca4972701c6f9a81cfd5e31ea3c85a732b9502aed455b2a9"} err="failed to get container status \"4485f7ec3044dc75ca4972701c6f9a81cfd5e31ea3c85a732b9502aed455b2a9\": rpc error: code = NotFound desc = could not find container \"4485f7ec3044dc75ca4972701c6f9a81cfd5e31ea3c85a732b9502aed455b2a9\": container with ID starting with 4485f7ec3044dc75ca4972701c6f9a81cfd5e31ea3c85a732b9502aed455b2a9 not found: ID does not exist"
Sep 17 03:45:13.803582 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.803526    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"]
Sep 17 03:45:13.806690 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:13.806666    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-57958f-55f67f868d-9bc22"]
Sep 17 03:45:14.118977 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:14.118947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:14.296943 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:14.296907    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b75164ca-32a6-4aa0-9b71-5cba18cde94b" path="/var/lib/kubelet/pods/b75164ca-32a6-4aa0-9b71-5cba18cde94b/volumes"
Sep 17 03:45:14.297294 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:14.297275    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c75c2f50-4470-4207-b6d2-ad67461396a4" path="/var/lib/kubelet/pods/c75c2f50-4470-4207-b6d2-ad67461396a4/volumes"
Sep 17 03:45:14.784935 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:14.784897    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:15.442038 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:15.441998    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:15.992714 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:15.992677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:16.024826 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.024790    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"]
Sep 17 03:45:16.025117 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.025087    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2" podUID="2c263c28-777c-43be-917e-d4c2a7712a1c" containerName="payload-processing" containerID="cri-o://e5d55ee6da3c6177e3b98383e736ce15711a07219253640eea69b2feaece4c91" gracePeriod=30
Sep 17 03:45:16.088631 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.088599    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:16.188545 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.188511    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:16.263360 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.263223    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:16.304743 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.304705    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"
Sep 17 03:45:16.363077 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.363046    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhjz4\" (UniqueName: \"kubernetes.io/projected/2c263c28-777c-43be-917e-d4c2a7712a1c-kube-api-access-nhjz4\") pod \"2c263c28-777c-43be-917e-d4c2a7712a1c\" (UID: \"2c263c28-777c-43be-917e-d4c2a7712a1c\") "
Sep 17 03:45:16.363260 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.363174    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c263c28-777c-43be-917e-d4c2a7712a1c-plugins-config-volume\") pod \"2c263c28-777c-43be-917e-d4c2a7712a1c\" (UID: \"2c263c28-777c-43be-917e-d4c2a7712a1c\") "
Sep 17 03:45:16.363639 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.363614    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c263c28-777c-43be-917e-d4c2a7712a1c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2c263c28-777c-43be-917e-d4c2a7712a1c" (UID: "2c263c28-777c-43be-917e-d4c2a7712a1c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:45:16.365145 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.365121    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c263c28-777c-43be-917e-d4c2a7712a1c-kube-api-access-nhjz4" (OuterVolumeSpecName: "kube-api-access-nhjz4") pod "2c263c28-777c-43be-917e-d4c2a7712a1c" (UID: "2c263c28-777c-43be-917e-d4c2a7712a1c"). InnerVolumeSpecName "kube-api-access-nhjz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:45:16.464242 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.464208    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c263c28-777c-43be-917e-d4c2a7712a1c-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:16.464242 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.464236    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhjz4\" (UniqueName: \"kubernetes.io/projected/2c263c28-777c-43be-917e-d4c2a7712a1c-kube-api-access-nhjz4\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:16.595323 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.595292    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:16.803421 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.803390    2576 generic.go:356] "Generic (PLEG): container finished" podID="2c263c28-777c-43be-917e-d4c2a7712a1c" containerID="e5d55ee6da3c6177e3b98383e736ce15711a07219253640eea69b2feaece4c91" exitCode=0
Sep 17 03:45:16.803610 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.803459    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"
Sep 17 03:45:16.803610 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.803472    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2" event={"ID":"2c263c28-777c-43be-917e-d4c2a7712a1c","Type":"ContainerDied","Data":"e5d55ee6da3c6177e3b98383e736ce15711a07219253640eea69b2feaece4c91"}
Sep 17 03:45:16.803610 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.803521    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2" event={"ID":"2c263c28-777c-43be-917e-d4c2a7712a1c","Type":"ContainerDied","Data":"8927eaec860d9e656acb5223a952275693cd3a4d38303d02012c0a1388a4428e"}
Sep 17 03:45:16.803610 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.803542    2576 scope.go:117] "RemoveContainer" containerID="e5d55ee6da3c6177e3b98383e736ce15711a07219253640eea69b2feaece4c91"
Sep 17 03:45:16.812939 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.812921    2576 scope.go:117] "RemoveContainer" containerID="e5d55ee6da3c6177e3b98383e736ce15711a07219253640eea69b2feaece4c91"
Sep 17 03:45:16.813235 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:45:16.813213    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5d55ee6da3c6177e3b98383e736ce15711a07219253640eea69b2feaece4c91\": container with ID starting with e5d55ee6da3c6177e3b98383e736ce15711a07219253640eea69b2feaece4c91 not found: ID does not exist" containerID="e5d55ee6da3c6177e3b98383e736ce15711a07219253640eea69b2feaece4c91"
Sep 17 03:45:16.813313 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.813245    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5d55ee6da3c6177e3b98383e736ce15711a07219253640eea69b2feaece4c91"} err="failed to get container status \"e5d55ee6da3c6177e3b98383e736ce15711a07219253640eea69b2feaece4c91\": rpc error: code = NotFound desc = could not find container \"e5d55ee6da3c6177e3b98383e736ce15711a07219253640eea69b2feaece4c91\": container with ID starting with e5d55ee6da3c6177e3b98383e736ce15711a07219253640eea69b2feaece4c91 not found: ID does not exist"
Sep 17 03:45:16.824552 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.824515    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"]
Sep 17 03:45:16.828631 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:16.828609    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-214fea-98f69797c-hmpm2"]
Sep 17 03:45:17.120912 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:17.120876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:17.766482 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:17.766422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:18.206885 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.206844    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb" podUID="8db5a26f-43bd-4c0a-88cd-630d7ebcac64" containerName="main" containerID="cri-o://11af2ebaa64e31f6f883cb2a16c67fb36aab6bea03c8bae03e4e650f6125b52b" gracePeriod=45
Sep 17 03:45:18.208420 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.208392    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5" podUID="7c845dae-d21b-48c3-a45a-81eaecab1847" containerName="main" containerID="cri-o://e19b8a7603f55b6559e94626fa7835cbc850f0ca00a2eb622db4579c9ae2c997" gracePeriod=45
Sep 17 03:45:18.294844 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.294815    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c263c28-777c-43be-917e-d4c2a7712a1c" path="/var/lib/kubelet/pods/2c263c28-777c-43be-917e-d4c2a7712a1c/volumes"
Sep 17 03:45:18.486066 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.486033    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:45:18.520061 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.520032    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:45:18.543724 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.543691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:18.582348 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.582317    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-home\") pod \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") "
Sep 17 03:45:18.582631 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.582365    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-tmp-dir\") pod \"7c845dae-d21b-48c3-a45a-81eaecab1847\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") "
Sep 17 03:45:18.582631 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.582420    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-tmp-dir\") pod \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") "
Sep 17 03:45:18.582631 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.582577    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-model-cache\") pod \"7c845dae-d21b-48c3-a45a-81eaecab1847\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") "
Sep 17 03:45:18.582631 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.582608    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-model-cache\") pod \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") "
Sep 17 03:45:18.582948 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.582931    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-dshm\") pod \"7c845dae-d21b-48c3-a45a-81eaecab1847\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") "
Sep 17 03:45:18.582948 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.582684    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7c845dae-d21b-48c3-a45a-81eaecab1847" (UID: "7c845dae-d21b-48c3-a45a-81eaecab1847"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:45:18.583067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.582684    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-home" (OuterVolumeSpecName: "home") pod "8db5a26f-43bd-4c0a-88cd-630d7ebcac64" (UID: "8db5a26f-43bd-4c0a-88cd-630d7ebcac64"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:45:18.583067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.582963    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sc988\" (UniqueName: \"kubernetes.io/projected/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-kube-api-access-sc988\") pod \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") "
Sep 17 03:45:18.583067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.583007    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2247v\" (UniqueName: \"kubernetes.io/projected/7c845dae-d21b-48c3-a45a-81eaecab1847-kube-api-access-2247v\") pod \"7c845dae-d21b-48c3-a45a-81eaecab1847\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") "
Sep 17 03:45:18.583067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.582719    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8db5a26f-43bd-4c0a-88cd-630d7ebcac64" (UID: "8db5a26f-43bd-4c0a-88cd-630d7ebcac64"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:45:18.583067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.583035    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-dshm\") pod \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") "
Sep 17 03:45:18.583067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.582822    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-model-cache" (OuterVolumeSpecName: "model-cache") pod "8db5a26f-43bd-4c0a-88cd-630d7ebcac64" (UID: "8db5a26f-43bd-4c0a-88cd-630d7ebcac64"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:45:18.583067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.582843    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-model-cache" (OuterVolumeSpecName: "model-cache") pod "7c845dae-d21b-48c3-a45a-81eaecab1847" (UID: "7c845dae-d21b-48c3-a45a-81eaecab1847"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:45:18.583067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.583063    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-tls-certs\") pod \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\" (UID: \"8db5a26f-43bd-4c0a-88cd-630d7ebcac64\") "
Sep 17 03:45:18.583466 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.583123    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c845dae-d21b-48c3-a45a-81eaecab1847-tls-certs\") pod \"7c845dae-d21b-48c3-a45a-81eaecab1847\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") "
Sep 17 03:45:18.583466 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.583150    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-home\") pod \"7c845dae-d21b-48c3-a45a-81eaecab1847\" (UID: \"7c845dae-d21b-48c3-a45a-81eaecab1847\") "
Sep 17 03:45:18.583466 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.583423    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-home\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:18.583466 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.583457    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-tmp-dir\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:18.583651 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.583470    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-tmp-dir\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:18.583651 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.583482    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-model-cache\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:18.583651 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.583494    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-model-cache\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:18.583651 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.583629    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-home" (OuterVolumeSpecName: "home") pod "7c845dae-d21b-48c3-a45a-81eaecab1847" (UID: "7c845dae-d21b-48c3-a45a-81eaecab1847"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:45:18.585686 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.585647    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-kube-api-access-sc988" (OuterVolumeSpecName: "kube-api-access-sc988") pod "8db5a26f-43bd-4c0a-88cd-630d7ebcac64" (UID: "8db5a26f-43bd-4c0a-88cd-630d7ebcac64"). InnerVolumeSpecName "kube-api-access-sc988". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:45:18.585918 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.585891    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-dshm" (OuterVolumeSpecName: "dshm") pod "8db5a26f-43bd-4c0a-88cd-630d7ebcac64" (UID: "8db5a26f-43bd-4c0a-88cd-630d7ebcac64"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:45:18.586059 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.586040    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c845dae-d21b-48c3-a45a-81eaecab1847-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7c845dae-d21b-48c3-a45a-81eaecab1847" (UID: "7c845dae-d21b-48c3-a45a-81eaecab1847"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:45:18.586298 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.586281    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c845dae-d21b-48c3-a45a-81eaecab1847-kube-api-access-2247v" (OuterVolumeSpecName: "kube-api-access-2247v") pod "7c845dae-d21b-48c3-a45a-81eaecab1847" (UID: "7c845dae-d21b-48c3-a45a-81eaecab1847"). InnerVolumeSpecName "kube-api-access-2247v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:45:18.586814 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.586791    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8db5a26f-43bd-4c0a-88cd-630d7ebcac64" (UID: "8db5a26f-43bd-4c0a-88cd-630d7ebcac64"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:45:18.587065 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.587041    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-dshm" (OuterVolumeSpecName: "dshm") pod "7c845dae-d21b-48c3-a45a-81eaecab1847" (UID: "7c845dae-d21b-48c3-a45a-81eaecab1847"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:45:18.684110 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.684071    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2247v\" (UniqueName: \"kubernetes.io/projected/7c845dae-d21b-48c3-a45a-81eaecab1847-kube-api-access-2247v\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:18.684110 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.684103    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-dshm\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:18.684110 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.684114    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-tls-certs\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:18.684349 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.684123    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c845dae-d21b-48c3-a45a-81eaecab1847-tls-certs\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:18.684349 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.684136    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-home\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:18.684349 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.684147    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c845dae-d21b-48c3-a45a-81eaecab1847-dshm\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:18.684349 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.684155    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sc988\" (UniqueName: \"kubernetes.io/projected/8db5a26f-43bd-4c0a-88cd-630d7ebcac64-kube-api-access-sc988\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:18.729732 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.729641    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:45:18.823620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.819561    2576 generic.go:356] "Generic (PLEG): container finished" podID="7c845dae-d21b-48c3-a45a-81eaecab1847" containerID="e19b8a7603f55b6559e94626fa7835cbc850f0ca00a2eb622db4579c9ae2c997" exitCode=0
Sep 17 03:45:18.823620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.819741    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5" event={"ID":"7c845dae-d21b-48c3-a45a-81eaecab1847","Type":"ContainerDied","Data":"e19b8a7603f55b6559e94626fa7835cbc850f0ca00a2eb622db4579c9ae2c997"}
Sep 17 03:45:18.823620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.819776    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5" event={"ID":"7c845dae-d21b-48c3-a45a-81eaecab1847","Type":"ContainerDied","Data":"8bccf3b4f2add5808cba20ed21576bb3f57e9cbe666a0b4aa1f638f109033eb7"}
Sep 17 03:45:18.823620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.819797    2576 scope.go:117] "RemoveContainer" containerID="e19b8a7603f55b6559e94626fa7835cbc850f0ca00a2eb622db4579c9ae2c997"
Sep 17 03:45:18.823620 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.819938    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"
Sep 17 03:45:18.826312 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.826277    2576 generic.go:356] "Generic (PLEG): container finished" podID="8db5a26f-43bd-4c0a-88cd-630d7ebcac64" containerID="11af2ebaa64e31f6f883cb2a16c67fb36aab6bea03c8bae03e4e650f6125b52b" exitCode=0
Sep 17 03:45:18.826490 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.826352    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb" event={"ID":"8db5a26f-43bd-4c0a-88cd-630d7ebcac64","Type":"ContainerDied","Data":"11af2ebaa64e31f6f883cb2a16c67fb36aab6bea03c8bae03e4e650f6125b52b"}
Sep 17 03:45:18.826490 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.826387    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb" event={"ID":"8db5a26f-43bd-4c0a-88cd-630d7ebcac64","Type":"ContainerDied","Data":"2463c27d1305920d143d3d73b0e53c1e086d1d257de86142ce19a76a107f1f74"}
Sep 17 03:45:18.826490 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.826475    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"
Sep 17 03:45:18.835370 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.835352    2576 scope.go:117] "RemoveContainer" containerID="e19b8a7603f55b6559e94626fa7835cbc850f0ca00a2eb622db4579c9ae2c997"
Sep 17 03:45:18.835692 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:45:18.835673    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e19b8a7603f55b6559e94626fa7835cbc850f0ca00a2eb622db4579c9ae2c997\": container with ID starting with e19b8a7603f55b6559e94626fa7835cbc850f0ca00a2eb622db4579c9ae2c997 not found: ID does not exist" containerID="e19b8a7603f55b6559e94626fa7835cbc850f0ca00a2eb622db4579c9ae2c997"
Sep 17 03:45:18.835744 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.835701    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19b8a7603f55b6559e94626fa7835cbc850f0ca00a2eb622db4579c9ae2c997"} err="failed to get container status \"e19b8a7603f55b6559e94626fa7835cbc850f0ca00a2eb622db4579c9ae2c997\": rpc error: code = NotFound desc = could not find container \"e19b8a7603f55b6559e94626fa7835cbc850f0ca00a2eb622db4579c9ae2c997\": container with ID starting with e19b8a7603f55b6559e94626fa7835cbc850f0ca00a2eb622db4579c9ae2c997 not found: ID does not exist"
Sep 17 03:45:18.835744 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.835720    2576 scope.go:117] "RemoveContainer" containerID="11af2ebaa64e31f6f883cb2a16c67fb36aab6bea03c8bae03e4e650f6125b52b"
Sep 17 03:45:18.843884 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.843866    2576 scope.go:117] "RemoveContainer" containerID="11af2ebaa64e31f6f883cb2a16c67fb36aab6bea03c8bae03e4e650f6125b52b"
Sep 17 03:45:18.844157 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:45:18.844137    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11af2ebaa64e31f6f883cb2a16c67fb36aab6bea03c8bae03e4e650f6125b52b\": container with ID starting with 11af2ebaa64e31f6f883cb2a16c67fb36aab6bea03c8bae03e4e650f6125b52b not found: ID does not exist" containerID="11af2ebaa64e31f6f883cb2a16c67fb36aab6bea03c8bae03e4e650f6125b52b"
Sep 17 03:45:18.844240 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.844165    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11af2ebaa64e31f6f883cb2a16c67fb36aab6bea03c8bae03e4e650f6125b52b"} err="failed to get container status \"11af2ebaa64e31f6f883cb2a16c67fb36aab6bea03c8bae03e4e650f6125b52b\": rpc error: code = NotFound desc = could not find container \"11af2ebaa64e31f6f883cb2a16c67fb36aab6bea03c8bae03e4e650f6125b52b\": container with ID starting with 11af2ebaa64e31f6f883cb2a16c67fb36aab6bea03c8bae03e4e650f6125b52b not found: ID does not exist"
Sep 17 03:45:18.845406 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.845386    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"]
Sep 17 03:45:18.848788 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.848769    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/distinct-w4-214fea-kserve-77f49b7446-4p4p5"]
Sep 17 03:45:18.858050 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.858027    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"]
Sep 17 03:45:18.862738 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:18.862720    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-214fea/distinct-2-w4-214fea-kserve-86cd5f89f7-s6sdb"]
Sep 17 03:45:19.697580 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:19.697550    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:19.797273 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:19.797238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:19.952602 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:19.952516    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:20.294755 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:20.294679    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c845dae-d21b-48c3-a45a-81eaecab1847" path="/var/lib/kubelet/pods/7c845dae-d21b-48c3-a45a-81eaecab1847/volumes"
Sep 17 03:45:20.295102 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:20.295087    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8db5a26f-43bd-4c0a-88cd-630d7ebcac64" path="/var/lib/kubelet/pods/8db5a26f-43bd-4c0a-88cd-630d7ebcac64/volumes"
Sep 17 03:45:20.421273 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:20.421224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:21.378988 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:21.378955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:22.070296 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:22.070265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:22.618228 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:22.618198    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:22.770555 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:22.770522    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:22.877861 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:22.877770    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:22.971952 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:22.971924    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:23.453076 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:23.453043    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:30.748699 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:30.748653    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"]
Sep 17 03:45:30.749247 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:30.748894    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4" podUID="d8f4c72c-cf3d-4e79-98a5-04948958dc60" containerName="maas-api" containerID="cri-o://c2c9d33ece956490688133f5c5c4b3286ce7921056acbd2fe559c752157f593f" gracePeriod=30
Sep 17 03:45:30.777717 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:30.777680    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:30.808371 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:30.808337    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"]
Sep 17 03:45:30.808645 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:30.808617    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n" podUID="b01d3c12-8dc6-42da-a6ad-df793133628e" containerName="payload-processing" containerID="cri-o://1d69b06f3bf0894b7ce16b3c7e393d7bebef72ff18386a0e77c5c587810e6029" gracePeriod=30
Sep 17 03:45:30.815366 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:30.815334    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"]
Sep 17 03:45:30.815612 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:30.815587    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5" podUID="cb91a032-966e-4f3e-81ad-aeb6decaf99d" containerName="payload-pre-processing" containerID="cri-o://17be8b73dbe221d120717e9e268516c22d043f80f1cab93aa2f1d9f49b4b9a1f" gracePeriod=30
Sep 17 03:45:30.849557 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:30.849515    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:30.902972 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:30.902933    2576 generic.go:356] "Generic (PLEG): container finished" podID="d8f4c72c-cf3d-4e79-98a5-04948958dc60" containerID="c2c9d33ece956490688133f5c5c4b3286ce7921056acbd2fe559c752157f593f" exitCode=0
Sep 17 03:45:30.903141 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:30.903000    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4" event={"ID":"d8f4c72c-cf3d-4e79-98a5-04948958dc60","Type":"ContainerDied","Data":"c2c9d33ece956490688133f5c5c4b3286ce7921056acbd2fe559c752157f593f"}
Sep 17 03:45:30.963143 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:30.963095    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:31.079284 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.079251    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:31.084320 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.084291    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:45:31.091182 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.091154    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"
Sep 17 03:45:31.124325 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.124296    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"
Sep 17 03:45:31.144636 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.144605    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:31.197672 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.197642    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6lr5\" (UniqueName: \"kubernetes.io/projected/d8f4c72c-cf3d-4e79-98a5-04948958dc60-kube-api-access-z6lr5\") pod \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") "
Sep 17 03:45:31.197672 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.197675    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b01d3c12-8dc6-42da-a6ad-df793133628e-plugins-config-volume\") pod \"b01d3c12-8dc6-42da-a6ad-df793133628e\" (UID: \"b01d3c12-8dc6-42da-a6ad-df793133628e\") "
Sep 17 03:45:31.197927 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.197707    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f4c72c-cf3d-4e79-98a5-04948958dc60-openshift-service-ca\") pod \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") "
Sep 17 03:45:31.197927 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.197722    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krqrk\" (UniqueName: \"kubernetes.io/projected/cb91a032-966e-4f3e-81ad-aeb6decaf99d-kube-api-access-krqrk\") pod \"cb91a032-966e-4f3e-81ad-aeb6decaf99d\" (UID: \"cb91a032-966e-4f3e-81ad-aeb6decaf99d\") "
Sep 17 03:45:31.197927 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.197750    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78lkx\" (UniqueName: \"kubernetes.io/projected/b01d3c12-8dc6-42da-a6ad-df793133628e-kube-api-access-78lkx\") pod \"b01d3c12-8dc6-42da-a6ad-df793133628e\" (UID: \"b01d3c12-8dc6-42da-a6ad-df793133628e\") "
Sep 17 03:45:31.197927 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.197792    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-maas-api-tls\") pod \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") "
Sep 17 03:45:31.197927 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.197854    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-metrics-certs\") pod \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\" (UID: \"d8f4c72c-cf3d-4e79-98a5-04948958dc60\") "
Sep 17 03:45:31.197927 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.197878    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb91a032-966e-4f3e-81ad-aeb6decaf99d-plugins-config-volume\") pod \"cb91a032-966e-4f3e-81ad-aeb6decaf99d\" (UID: \"cb91a032-966e-4f3e-81ad-aeb6decaf99d\") "
Sep 17 03:45:31.198261 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.198121    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b01d3c12-8dc6-42da-a6ad-df793133628e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b01d3c12-8dc6-42da-a6ad-df793133628e" (UID: "b01d3c12-8dc6-42da-a6ad-df793133628e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:45:31.198468 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.198408    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8f4c72c-cf3d-4e79-98a5-04948958dc60-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d8f4c72c-cf3d-4e79-98a5-04948958dc60" (UID: "d8f4c72c-cf3d-4e79-98a5-04948958dc60"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:45:31.198601 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.198493    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb91a032-966e-4f3e-81ad-aeb6decaf99d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cb91a032-966e-4f3e-81ad-aeb6decaf99d" (UID: "cb91a032-966e-4f3e-81ad-aeb6decaf99d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:45:31.200111 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.200084    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d8f4c72c-cf3d-4e79-98a5-04948958dc60" (UID: "d8f4c72c-cf3d-4e79-98a5-04948958dc60"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:45:31.200197 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.200112    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b01d3c12-8dc6-42da-a6ad-df793133628e-kube-api-access-78lkx" (OuterVolumeSpecName: "kube-api-access-78lkx") pod "b01d3c12-8dc6-42da-a6ad-df793133628e" (UID: "b01d3c12-8dc6-42da-a6ad-df793133628e"). InnerVolumeSpecName "kube-api-access-78lkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:45:31.200197 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.200142    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8f4c72c-cf3d-4e79-98a5-04948958dc60-kube-api-access-z6lr5" (OuterVolumeSpecName: "kube-api-access-z6lr5") pod "d8f4c72c-cf3d-4e79-98a5-04948958dc60" (UID: "d8f4c72c-cf3d-4e79-98a5-04948958dc60"). InnerVolumeSpecName "kube-api-access-z6lr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:45:31.200289 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.200269    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb91a032-966e-4f3e-81ad-aeb6decaf99d-kube-api-access-krqrk" (OuterVolumeSpecName: "kube-api-access-krqrk") pod "cb91a032-966e-4f3e-81ad-aeb6decaf99d" (UID: "cb91a032-966e-4f3e-81ad-aeb6decaf99d"). InnerVolumeSpecName "kube-api-access-krqrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:45:31.200344 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.200319    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d8f4c72c-cf3d-4e79-98a5-04948958dc60" (UID: "d8f4c72c-cf3d-4e79-98a5-04948958dc60"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:45:31.298791 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.298707    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78lkx\" (UniqueName: \"kubernetes.io/projected/b01d3c12-8dc6-42da-a6ad-df793133628e-kube-api-access-78lkx\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:31.298791 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.298737    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-maas-api-tls\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:31.298791 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.298746    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8f4c72c-cf3d-4e79-98a5-04948958dc60-metrics-certs\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:31.298791 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.298755    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb91a032-966e-4f3e-81ad-aeb6decaf99d-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:31.298791 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.298763    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6lr5\" (UniqueName: \"kubernetes.io/projected/d8f4c72c-cf3d-4e79-98a5-04948958dc60-kube-api-access-z6lr5\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:31.298791 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.298772    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b01d3c12-8dc6-42da-a6ad-df793133628e-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:31.298791 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.298780    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f4c72c-cf3d-4e79-98a5-04948958dc60-openshift-service-ca\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:31.298791 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.298788    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krqrk\" (UniqueName: \"kubernetes.io/projected/cb91a032-966e-4f3e-81ad-aeb6decaf99d-kube-api-access-krqrk\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:45:31.477183 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.477155    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:31.577301 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.577259    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:31.911397 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.911295    2576 generic.go:356] "Generic (PLEG): container finished" podID="cb91a032-966e-4f3e-81ad-aeb6decaf99d" containerID="17be8b73dbe221d120717e9e268516c22d043f80f1cab93aa2f1d9f49b4b9a1f" exitCode=0
Sep 17 03:45:31.911397 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.911361    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"
Sep 17 03:45:31.911929 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.911393    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5" event={"ID":"cb91a032-966e-4f3e-81ad-aeb6decaf99d","Type":"ContainerDied","Data":"17be8b73dbe221d120717e9e268516c22d043f80f1cab93aa2f1d9f49b4b9a1f"}
Sep 17 03:45:31.911929 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.911468    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5" event={"ID":"cb91a032-966e-4f3e-81ad-aeb6decaf99d","Type":"ContainerDied","Data":"ef7211124b57d226bc97bc74b76f0ac5c8bb03855f9f1e2a48d15a252342a3d9"}
Sep 17 03:45:31.911929 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.911503    2576 scope.go:117] "RemoveContainer" containerID="17be8b73dbe221d120717e9e268516c22d043f80f1cab93aa2f1d9f49b4b9a1f"
Sep 17 03:45:31.915704 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.914474    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"
Sep 17 03:45:31.915704 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.914502    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4" event={"ID":"d8f4c72c-cf3d-4e79-98a5-04948958dc60","Type":"ContainerDied","Data":"2d7946368067051b4d5421d04eae4f4ed3ebb07d0216ae34bb8054a3344b8e78"}
Sep 17 03:45:31.916203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.916174    2576 generic.go:356] "Generic (PLEG): container finished" podID="b01d3c12-8dc6-42da-a6ad-df793133628e" containerID="1d69b06f3bf0894b7ce16b3c7e393d7bebef72ff18386a0e77c5c587810e6029" exitCode=0
Sep 17 03:45:31.916260 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.916215    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"
Sep 17 03:45:31.916260 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.916227    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n" event={"ID":"b01d3c12-8dc6-42da-a6ad-df793133628e","Type":"ContainerDied","Data":"1d69b06f3bf0894b7ce16b3c7e393d7bebef72ff18386a0e77c5c587810e6029"}
Sep 17 03:45:31.916374 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.916264    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n" event={"ID":"b01d3c12-8dc6-42da-a6ad-df793133628e","Type":"ContainerDied","Data":"a90db0c3fd5c4c12467a4059bd3a6e15a2d7697e9fe46fdf01d5ef8ecffe038e"}
Sep 17 03:45:31.924267 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.924249    2576 scope.go:117] "RemoveContainer" containerID="17be8b73dbe221d120717e9e268516c22d043f80f1cab93aa2f1d9f49b4b9a1f"
Sep 17 03:45:31.924610 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:45:31.924583    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17be8b73dbe221d120717e9e268516c22d043f80f1cab93aa2f1d9f49b4b9a1f\": container with ID starting with 17be8b73dbe221d120717e9e268516c22d043f80f1cab93aa2f1d9f49b4b9a1f not found: ID does not exist" containerID="17be8b73dbe221d120717e9e268516c22d043f80f1cab93aa2f1d9f49b4b9a1f"
Sep 17 03:45:31.924724 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.924613    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17be8b73dbe221d120717e9e268516c22d043f80f1cab93aa2f1d9f49b4b9a1f"} err="failed to get container status \"17be8b73dbe221d120717e9e268516c22d043f80f1cab93aa2f1d9f49b4b9a1f\": rpc error: code = NotFound desc = could not find container \"17be8b73dbe221d120717e9e268516c22d043f80f1cab93aa2f1d9f49b4b9a1f\": container with ID starting with 17be8b73dbe221d120717e9e268516c22d043f80f1cab93aa2f1d9f49b4b9a1f not found: ID does not exist"
Sep 17 03:45:31.924724 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.924629    2576 scope.go:117] "RemoveContainer" containerID="c2c9d33ece956490688133f5c5c4b3286ce7921056acbd2fe559c752157f593f"
Sep 17 03:45:31.936557 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.936533    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"]
Sep 17 03:45:31.936800 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.936770    2576 scope.go:117] "RemoveContainer" containerID="1d69b06f3bf0894b7ce16b3c7e393d7bebef72ff18386a0e77c5c587810e6029"
Sep 17 03:45:31.943585 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.943552    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-fc820e-54cf885694-5rxv5"]
Sep 17 03:45:31.946304 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.946289    2576 scope.go:117] "RemoveContainer" containerID="1d69b06f3bf0894b7ce16b3c7e393d7bebef72ff18386a0e77c5c587810e6029"
Sep 17 03:45:31.946603 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:45:31.946581    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d69b06f3bf0894b7ce16b3c7e393d7bebef72ff18386a0e77c5c587810e6029\": container with ID starting with 1d69b06f3bf0894b7ce16b3c7e393d7bebef72ff18386a0e77c5c587810e6029 not found: ID does not exist" containerID="1d69b06f3bf0894b7ce16b3c7e393d7bebef72ff18386a0e77c5c587810e6029"
Sep 17 03:45:31.946690 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.946605    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d69b06f3bf0894b7ce16b3c7e393d7bebef72ff18386a0e77c5c587810e6029"} err="failed to get container status \"1d69b06f3bf0894b7ce16b3c7e393d7bebef72ff18386a0e77c5c587810e6029\": rpc error: code = NotFound desc = could not find container \"1d69b06f3bf0894b7ce16b3c7e393d7bebef72ff18386a0e77c5c587810e6029\": container with ID starting with 1d69b06f3bf0894b7ce16b3c7e393d7bebef72ff18386a0e77c5c587810e6029 not found: ID does not exist"
Sep 17 03:45:31.953718 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.953696    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"]
Sep 17 03:45:31.957547 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.957525    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4"]
Sep 17 03:45:31.966499 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.966464    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"]
Sep 17 03:45:31.968754 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:31.968734    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-fc820e-595799dd95-cmg6n"]
Sep 17 03:45:32.163337 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:32.163254    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:32.298866 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:32.298822    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b01d3c12-8dc6-42da-a6ad-df793133628e" path="/var/lib/kubelet/pods/b01d3c12-8dc6-42da-a6ad-df793133628e/volumes"
Sep 17 03:45:32.299407 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:32.299382    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb91a032-966e-4f3e-81ad-aeb6decaf99d" path="/var/lib/kubelet/pods/cb91a032-966e-4f3e-81ad-aeb6decaf99d/volumes"
Sep 17 03:45:32.300336 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:32.300302    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8f4c72c-cf3d-4e79-98a5-04948958dc60" path="/var/lib/kubelet/pods/d8f4c72c-cf3d-4e79-98a5-04948958dc60/volumes"
Sep 17 03:45:32.819519 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:32.819482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:34.025360 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:34.025299    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fc820e-7d5ccd595c-cdjg4" podUID="d8f4c72c-cf3d-4e79-98a5-04948958dc60" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.53:8443/health\": context deadline exceeded"
Sep 17 03:45:37.160865 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:37.160834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:37.220653 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:37.220625    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:37.333971 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:37.333932    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:45:37.413857 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:37.413775    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:39.391331 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:39.391299    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:39.451329 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:39.451301    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:40.283300 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:40.283266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:45.734554 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:45.734520    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:50.926328 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:50.926288    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:45:51.037823 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:51.037791    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:51.093160 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:51.093121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:56.559589 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:56.559553    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"]
Sep 17 03:45:56.586191 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:56.586162    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:57.498052 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:57.498024    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:45:59.935415 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:45:59.935383    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:46:05.244633 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:05.244598    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:46:09.435046 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:09.435014    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:46:11.580235 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.580187    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm" podUID="65348adf-214b-4933-a984-431523fa7b21" containerName="main" containerID="cri-o://3d94687287fa5eb86ab637e2178c904461500b00ac8e3d2ab532e414016763ec" gracePeriod=45
Sep 17 03:46:11.829143 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.829110    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:46:11.848169 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.848085    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-tmp-dir\") pod \"65348adf-214b-4933-a984-431523fa7b21\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") "
Sep 17 03:46:11.848169 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.848127    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-dshm\") pod \"65348adf-214b-4933-a984-431523fa7b21\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") "
Sep 17 03:46:11.848169 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.848145    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-model-cache\") pod \"65348adf-214b-4933-a984-431523fa7b21\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") "
Sep 17 03:46:11.848458 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.848174    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7djxw\" (UniqueName: \"kubernetes.io/projected/65348adf-214b-4933-a984-431523fa7b21-kube-api-access-7djxw\") pod \"65348adf-214b-4933-a984-431523fa7b21\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") "
Sep 17 03:46:11.848458 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.848216    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65348adf-214b-4933-a984-431523fa7b21-tls-certs\") pod \"65348adf-214b-4933-a984-431523fa7b21\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") "
Sep 17 03:46:11.848458 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.848247    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-home\") pod \"65348adf-214b-4933-a984-431523fa7b21\" (UID: \"65348adf-214b-4933-a984-431523fa7b21\") "
Sep 17 03:46:11.848635 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.848425    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-model-cache" (OuterVolumeSpecName: "model-cache") pod "65348adf-214b-4933-a984-431523fa7b21" (UID: "65348adf-214b-4933-a984-431523fa7b21"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:46:11.848635 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.848561    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "65348adf-214b-4933-a984-431523fa7b21" (UID: "65348adf-214b-4933-a984-431523fa7b21"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:46:11.848635 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.848585    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-model-cache\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:46:11.848801 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.848666    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-home" (OuterVolumeSpecName: "home") pod "65348adf-214b-4933-a984-431523fa7b21" (UID: "65348adf-214b-4933-a984-431523fa7b21"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:46:11.850502 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.850475    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-dshm" (OuterVolumeSpecName: "dshm") pod "65348adf-214b-4933-a984-431523fa7b21" (UID: "65348adf-214b-4933-a984-431523fa7b21"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:46:11.850611 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.850537    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65348adf-214b-4933-a984-431523fa7b21-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "65348adf-214b-4933-a984-431523fa7b21" (UID: "65348adf-214b-4933-a984-431523fa7b21"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:46:11.850611 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.850550    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65348adf-214b-4933-a984-431523fa7b21-kube-api-access-7djxw" (OuterVolumeSpecName: "kube-api-access-7djxw") pod "65348adf-214b-4933-a984-431523fa7b21" (UID: "65348adf-214b-4933-a984-431523fa7b21"). InnerVolumeSpecName "kube-api-access-7djxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:46:11.949174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.949123    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7djxw\" (UniqueName: \"kubernetes.io/projected/65348adf-214b-4933-a984-431523fa7b21-kube-api-access-7djxw\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:46:11.949174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.949170    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65348adf-214b-4933-a984-431523fa7b21-tls-certs\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:46:11.949174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.949179    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-home\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:46:11.949174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.949187    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-tmp-dir\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:46:11.949174 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:11.949195    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65348adf-214b-4933-a984-431523fa7b21-dshm\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:46:12.136810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:12.136715    2576 generic.go:356] "Generic (PLEG): container finished" podID="65348adf-214b-4933-a984-431523fa7b21" containerID="3d94687287fa5eb86ab637e2178c904461500b00ac8e3d2ab532e414016763ec" exitCode=0
Sep 17 03:46:12.136810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:12.136763    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"
Sep 17 03:46:12.136810 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:12.136794    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm" event={"ID":"65348adf-214b-4933-a984-431523fa7b21","Type":"ContainerDied","Data":"3d94687287fa5eb86ab637e2178c904461500b00ac8e3d2ab532e414016763ec"}
Sep 17 03:46:12.137039 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:12.136836    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm" event={"ID":"65348adf-214b-4933-a984-431523fa7b21","Type":"ContainerDied","Data":"9a2c90d2aaa682943f48337ac5f58c13fe87ff04eca92344382d9316d21a51fb"}
Sep 17 03:46:12.137039 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:12.136853    2576 scope.go:117] "RemoveContainer" containerID="3d94687287fa5eb86ab637e2178c904461500b00ac8e3d2ab532e414016763ec"
Sep 17 03:46:12.146228 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:12.146203    2576 scope.go:117] "RemoveContainer" containerID="3d94687287fa5eb86ab637e2178c904461500b00ac8e3d2ab532e414016763ec"
Sep 17 03:46:12.146533 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:46:12.146514    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d94687287fa5eb86ab637e2178c904461500b00ac8e3d2ab532e414016763ec\": container with ID starting with 3d94687287fa5eb86ab637e2178c904461500b00ac8e3d2ab532e414016763ec not found: ID does not exist" containerID="3d94687287fa5eb86ab637e2178c904461500b00ac8e3d2ab532e414016763ec"
Sep 17 03:46:12.146593 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:12.146542    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d94687287fa5eb86ab637e2178c904461500b00ac8e3d2ab532e414016763ec"} err="failed to get container status \"3d94687287fa5eb86ab637e2178c904461500b00ac8e3d2ab532e414016763ec\": rpc error: code = NotFound desc = could not find container \"3d94687287fa5eb86ab637e2178c904461500b00ac8e3d2ab532e414016763ec\": container with ID starting with 3d94687287fa5eb86ab637e2178c904461500b00ac8e3d2ab532e414016763ec not found: ID does not exist"
Sep 17 03:46:12.156681 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:12.156658    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"]
Sep 17 03:46:12.160953 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:12.160930    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-22e045/auth-test-model-22e045-kserve-67ddf895ff-97rpm"]
Sep 17 03:46:12.295444 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:12.295388    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65348adf-214b-4933-a984-431523fa7b21" path="/var/lib/kubelet/pods/65348adf-214b-4933-a984-431523fa7b21/volumes"
Sep 17 03:46:18.383234 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:18.383198    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:46:18.484117 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:18.484086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:46:46.069931 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:46.069893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:46:49.036736 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:46:49.036697    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:47:26.733112 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:47:26.733078    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:47:36.429482 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:47:36.429443    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:47:46.238328 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:47:46.238287    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:47:51.379478 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:47:51.379418    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:47:52.225018 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:47:52.224964    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:47:53.603168 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:47:53.603136    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:47:54.732627 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:47:54.732589    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:47:55.209612 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:47:55.209577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:48:05.144843 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:48:05.144804    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:48:32.819812 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:48:32.819754    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:48:36.332254 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:48:36.332219    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:48:49.742687 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:48:49.742650    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:48:56.839450 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:48:56.839393    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:00.551055 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:00.551018    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:49:05.680330 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:05.680299    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:06.575548 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:06.575505    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:06.998785 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:06.998700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:07.288038 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:07.287956    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:07.591783 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:07.591748    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:09.842788 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:09.842755    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:10.050451 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:10.050405    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:10.142515 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:10.142402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:10.622745 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:10.622710    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:11.371492 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.371460    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"]
Sep 17 03:49:11.371984 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.371948    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb91a032-966e-4f3e-81ad-aeb6decaf99d" containerName="payload-pre-processing"
Sep 17 03:49:11.371984 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.371973    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb91a032-966e-4f3e-81ad-aeb6decaf99d" containerName="payload-pre-processing"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.371987    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c845dae-d21b-48c3-a45a-81eaecab1847" containerName="main"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.371996    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c845dae-d21b-48c3-a45a-81eaecab1847" containerName="main"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372009    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8f4c72c-cf3d-4e79-98a5-04948958dc60" containerName="maas-api"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372017    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8f4c72c-cf3d-4e79-98a5-04948958dc60" containerName="maas-api"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372027    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b01d3c12-8dc6-42da-a6ad-df793133628e" containerName="payload-processing"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372035    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b01d3c12-8dc6-42da-a6ad-df793133628e" containerName="payload-processing"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372052    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c75c2f50-4470-4207-b6d2-ad67461396a4" containerName="maas-api"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372060    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c75c2f50-4470-4207-b6d2-ad67461396a4" containerName="maas-api"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372075    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c263c28-777c-43be-917e-d4c2a7712a1c" containerName="payload-processing"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372083    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c263c28-777c-43be-917e-d4c2a7712a1c" containerName="payload-processing"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372093    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65348adf-214b-4933-a984-431523fa7b21" containerName="main"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372101    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="65348adf-214b-4933-a984-431523fa7b21" containerName="main"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372115    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db5a26f-43bd-4c0a-88cd-630d7ebcac64" containerName="main"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372126    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db5a26f-43bd-4c0a-88cd-630d7ebcac64" containerName="main"
Sep 17 03:49:11.372134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372137    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b75164ca-32a6-4aa0-9b71-5cba18cde94b" containerName="cleanup"
Sep 17 03:49:11.372870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372145    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b75164ca-32a6-4aa0-9b71-5cba18cde94b" containerName="cleanup"
Sep 17 03:49:11.372870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372234    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb91a032-966e-4f3e-81ad-aeb6decaf99d" containerName="payload-pre-processing"
Sep 17 03:49:11.372870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372246    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8db5a26f-43bd-4c0a-88cd-630d7ebcac64" containerName="main"
Sep 17 03:49:11.372870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372257    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b01d3c12-8dc6-42da-a6ad-df793133628e" containerName="payload-processing"
Sep 17 03:49:11.372870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372268    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b75164ca-32a6-4aa0-9b71-5cba18cde94b" containerName="cleanup"
Sep 17 03:49:11.372870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372278    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c75c2f50-4470-4207-b6d2-ad67461396a4" containerName="maas-api"
Sep 17 03:49:11.372870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372287    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c845dae-d21b-48c3-a45a-81eaecab1847" containerName="main"
Sep 17 03:49:11.372870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372296    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c263c28-777c-43be-917e-d4c2a7712a1c" containerName="payload-processing"
Sep 17 03:49:11.372870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372305    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8f4c72c-cf3d-4e79-98a5-04948958dc60" containerName="maas-api"
Sep 17 03:49:11.372870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.372314    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="65348adf-214b-4933-a984-431523fa7b21" containerName="main"
Sep 17 03:49:11.375690 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.375665    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.377916 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.377895    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-d0ab1a-serving-cert\""
Sep 17 03:49:11.386638 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.386579    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"]
Sep 17 03:49:11.404574 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.404547    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:11.414713 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.414686    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15d03a6f-2ea1-4e27-9274-f45170ad095c-metrics-certs\") pod \"maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.414878 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.414727    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15d03a6f-2ea1-4e27-9274-f45170ad095c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.414878 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.414751    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tlq5\" (UniqueName: \"kubernetes.io/projected/15d03a6f-2ea1-4e27-9274-f45170ad095c-kube-api-access-9tlq5\") pod \"maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.414988 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.414867    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15d03a6f-2ea1-4e27-9274-f45170ad095c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.505015 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.504975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:11.515849 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.515818    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15d03a6f-2ea1-4e27-9274-f45170ad095c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.516020 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.515861    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tlq5\" (UniqueName: \"kubernetes.io/projected/15d03a6f-2ea1-4e27-9274-f45170ad095c-kube-api-access-9tlq5\") pod \"maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.516020 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.515913    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15d03a6f-2ea1-4e27-9274-f45170ad095c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.516020 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.515983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15d03a6f-2ea1-4e27-9274-f45170ad095c-metrics-certs\") pod \"maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.523088 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.517038    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15d03a6f-2ea1-4e27-9274-f45170ad095c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.524232 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.524202    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15d03a6f-2ea1-4e27-9274-f45170ad095c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.525087 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.525067    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15d03a6f-2ea1-4e27-9274-f45170ad095c-metrics-certs\") pod \"maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.525212 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.525190    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tlq5\" (UniqueName: \"kubernetes.io/projected/15d03a6f-2ea1-4e27-9274-f45170ad095c-kube-api-access-9tlq5\") pod \"maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.563894 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.563863    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:11.687705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.687594    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:11.877785 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.877744    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:11.968532 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.968503    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"]
Sep 17 03:49:11.970297 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:49:11.970266    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d03a6f_2ea1_4e27_9274_f45170ad095c.slice/crio-c4df0c8401a9869bf7dbdfb900493bceacd8b8fed792e1ee5d09a21da6595b78 WatchSource:0}: Error finding container c4df0c8401a9869bf7dbdfb900493bceacd8b8fed792e1ee5d09a21da6595b78: Status 404 returned error can't find the container with id c4df0c8401a9869bf7dbdfb900493bceacd8b8fed792e1ee5d09a21da6595b78
Sep 17 03:49:11.971935 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.971919    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 03:49:11.974104 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:11.974078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:12.015691 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.015653    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"]
Sep 17 03:49:12.019983 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.019967    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"
Sep 17 03:49:12.022133 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.022111    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-d0ab1a-dockercfg-cvjft\""
Sep 17 03:49:12.022133 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.022113    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-d0ab1a\""
Sep 17 03:49:12.029978 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.029957    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"]
Sep 17 03:49:12.098205 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.098172    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7" event={"ID":"15d03a6f-2ea1-4e27-9274-f45170ad095c","Type":"ContainerStarted","Data":"c4df0c8401a9869bf7dbdfb900493bceacd8b8fed792e1ee5d09a21da6595b78"}
Sep 17 03:49:12.120896 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.120863    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f775d34-3f35-492e-8a3f-ed05684ac8cd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5\" (UID: \"1f775d34-3f35-492e-8a3f-ed05684ac8cd\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"
Sep 17 03:49:12.121091 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.120904    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l67tq\" (UniqueName: \"kubernetes.io/projected/1f775d34-3f35-492e-8a3f-ed05684ac8cd-kube-api-access-l67tq\") pod \"payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5\" (UID: \"1f775d34-3f35-492e-8a3f-ed05684ac8cd\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"
Sep 17 03:49:12.222377 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.222287    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f775d34-3f35-492e-8a3f-ed05684ac8cd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5\" (UID: \"1f775d34-3f35-492e-8a3f-ed05684ac8cd\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"
Sep 17 03:49:12.222377 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.222328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l67tq\" (UniqueName: \"kubernetes.io/projected/1f775d34-3f35-492e-8a3f-ed05684ac8cd-kube-api-access-l67tq\") pod \"payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5\" (UID: \"1f775d34-3f35-492e-8a3f-ed05684ac8cd\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"
Sep 17 03:49:12.223178 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.223155    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f775d34-3f35-492e-8a3f-ed05684ac8cd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5\" (UID: \"1f775d34-3f35-492e-8a3f-ed05684ac8cd\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"
Sep 17 03:49:12.230522 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.230499    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l67tq\" (UniqueName: \"kubernetes.io/projected/1f775d34-3f35-492e-8a3f-ed05684ac8cd-kube-api-access-l67tq\") pod \"payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5\" (UID: \"1f775d34-3f35-492e-8a3f-ed05684ac8cd\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"
Sep 17 03:49:12.331406 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.331364    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"
Sep 17 03:49:12.605087 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:12.605060    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"]
Sep 17 03:49:12.607472 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:49:12.607440    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f775d34_3f35_492e_8a3f_ed05684ac8cd.slice/crio-822fb25226f975050faca247871efde69ef76da9611d5cfc6dc99414895c7fd5 WatchSource:0}: Error finding container 822fb25226f975050faca247871efde69ef76da9611d5cfc6dc99414895c7fd5: Status 404 returned error can't find the container with id 822fb25226f975050faca247871efde69ef76da9611d5cfc6dc99414895c7fd5
Sep 17 03:49:13.105907 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:13.105858    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7" event={"ID":"15d03a6f-2ea1-4e27-9274-f45170ad095c","Type":"ContainerStarted","Data":"85fde3e927d9b90062294f4c16b6141a0830cdadc49ab423df5b2872e1940c21"}
Sep 17 03:49:13.106295 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:13.105960    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:13.107705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:13.107669    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5" event={"ID":"1f775d34-3f35-492e-8a3f-ed05684ac8cd","Type":"ContainerStarted","Data":"f5139fe5bf033e7e9409177430e03bf29ddd8bc414e68cf9f7a2137fe7f8ceb9"}
Sep 17 03:49:13.107840 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:13.107729    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5" event={"ID":"1f775d34-3f35-492e-8a3f-ed05684ac8cd","Type":"ContainerStarted","Data":"822fb25226f975050faca247871efde69ef76da9611d5cfc6dc99414895c7fd5"}
Sep 17 03:49:13.107840 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:13.107776    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"
Sep 17 03:49:13.123045 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:13.122994    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7" podStartSLOduration=1.734368496 podStartE2EDuration="2.122977972s" totalImagesPullingTime="388.609476ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:49:13.122240297 +0000 UTC m=+1781.466966376" watchObservedRunningTime="2026-09-17 03:49:13.122977972 +0000 UTC m=+1781.467704050"
Sep 17 03:49:13.137324 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:13.137275    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5" podStartSLOduration=2.1372649089999998 podStartE2EDuration="2.137264909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:49:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:49:13.135965567 +0000 UTC m=+1781.480691647" watchObservedRunningTime="2026-09-17 03:49:13.137264909 +0000 UTC m=+1781.481990988"
Sep 17 03:49:15.339362 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:15.339321    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:49:19.119478 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:19.119425    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:49:24.116267 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:24.116187    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"
Sep 17 03:49:32.348367 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:32.348340    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:49:32.351150 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:32.351124    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:49:32.526125 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:32.526089    2576 scope.go:117] "RemoveContainer" containerID="c2c23cad8e8d434a4a96fd10677b836faf3005bb8df1fd0207110a3842bafbc8"
Sep 17 03:49:34.203206 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:34.203171    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:34.971655 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:34.971619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:35.074335 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:35.074301    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:35.636672 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:35.636638    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:35.736833 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:35.736780    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:35.984142 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:35.984035    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:36.222111 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:36.222074    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:36.408709 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:36.408676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:38.008652 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:38.008618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:38.792635 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:38.792597    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:38.889606 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:38.889575    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:39.609092 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:39.609056    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:40.269232 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:40.269195    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:40.930863 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:40.930782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:41.032995 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.032959    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:41.134156 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.134121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:41.375657 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.375627    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:41.476349 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.476305    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:41.564639 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.564601    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"]
Sep 17 03:49:41.568078 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.568056    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"
Sep 17 03:49:41.570466 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.570417    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-84cd5d\""
Sep 17 03:49:41.570612 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.570499    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-84cd5d-dockercfg-wrkx6\""
Sep 17 03:49:41.576280 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.576073    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"]
Sep 17 03:49:41.682031 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.681930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23cee20c-e748-4681-872f-1b6c2bba21b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t\" (UID: \"23cee20c-e748-4681-872f-1b6c2bba21b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"
Sep 17 03:49:41.682031 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.682024    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58x2p\" (UniqueName: \"kubernetes.io/projected/23cee20c-e748-4681-872f-1b6c2bba21b5-kube-api-access-58x2p\") pod \"payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t\" (UID: \"23cee20c-e748-4681-872f-1b6c2bba21b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"
Sep 17 03:49:41.783649 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.783609    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23cee20c-e748-4681-872f-1b6c2bba21b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t\" (UID: \"23cee20c-e748-4681-872f-1b6c2bba21b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"
Sep 17 03:49:41.783649 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.783657    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58x2p\" (UniqueName: \"kubernetes.io/projected/23cee20c-e748-4681-872f-1b6c2bba21b5-kube-api-access-58x2p\") pod \"payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t\" (UID: \"23cee20c-e748-4681-872f-1b6c2bba21b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"
Sep 17 03:49:41.784410 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.784385    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23cee20c-e748-4681-872f-1b6c2bba21b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t\" (UID: \"23cee20c-e748-4681-872f-1b6c2bba21b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"
Sep 17 03:49:41.791067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.791039    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58x2p\" (UniqueName: \"kubernetes.io/projected/23cee20c-e748-4681-872f-1b6c2bba21b5-kube-api-access-58x2p\") pod \"payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t\" (UID: \"23cee20c-e748-4681-872f-1b6c2bba21b5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"
Sep 17 03:49:41.880112 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:41.880071    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"
Sep 17 03:49:42.153992 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:42.153963    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"]
Sep 17 03:49:42.155683 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:49:42.155650    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23cee20c_e748_4681_872f_1b6c2bba21b5.slice/crio-5a18f06579e1ae1a80214adf64fb5d6c3b8fcfc910e69496c7e9f2a834461342 WatchSource:0}: Error finding container 5a18f06579e1ae1a80214adf64fb5d6c3b8fcfc910e69496c7e9f2a834461342: Status 404 returned error can't find the container with id 5a18f06579e1ae1a80214adf64fb5d6c3b8fcfc910e69496c7e9f2a834461342
Sep 17 03:49:42.267868 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:42.267833    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t" event={"ID":"23cee20c-e748-4681-872f-1b6c2bba21b5","Type":"ContainerStarted","Data":"70a1d56a77787d160e0ca1776b45e151c181f10f887825650ad985df50afea51"}
Sep 17 03:49:42.268048 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:42.267888    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t" event={"ID":"23cee20c-e748-4681-872f-1b6c2bba21b5","Type":"ContainerStarted","Data":"5a18f06579e1ae1a80214adf64fb5d6c3b8fcfc910e69496c7e9f2a834461342"}
Sep 17 03:49:42.268048 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:42.267909    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"
Sep 17 03:49:42.284709 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:42.284648    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t" podStartSLOduration=1.284631327 podStartE2EDuration="1.284631327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:49:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:49:42.281909646 +0000 UTC m=+1810.626635737" watchObservedRunningTime="2026-09-17 03:49:42.284631327 +0000 UTC m=+1810.629357405"
Sep 17 03:49:43.337668 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:43.337631    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:49:46.042038 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:46.042001    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:49:53.275939 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:53.275910    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"
Sep 17 03:49:58.088121 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:49:58.088088    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:01.797001 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:01.796966    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:03.082403 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:03.082370    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:03.740227 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:03.740196    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:03.838814 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:03.838758    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:04.490857 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:04.490815    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:05.554138 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:05.554104    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:06.126930 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:06.126900    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:08.323966 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:08.323926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:22.607790 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:22.607757    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:26.540534 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:26.540496    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:50:32.541681 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:32.541651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:32.643113 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:32.643057    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:43.837831 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:43.837793    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:50:50.163729 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:50.163678    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5_1f775d34-3f35-492e-8a3f-ed05684ac8cd/payload-processing/0.log"
Sep 17 03:50:50.826257 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:50.826209    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:51.151278 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:51.151151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:51.410302 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:51.410199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:53.492781 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:53.492749    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:54.171114 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:54.171025    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:54.319159 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:54.319123    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:54.427269 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:54.427168    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:55.215062 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:55.215025    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:56.000090 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:56.000048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:56.654870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:56.654836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:57.421037 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:57.421000    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:57.522288 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:57.522244    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:57.923247 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:57.923210    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:58.020618 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.020590    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:58.081878 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.081848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:50:58.084980 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.084953    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"]
Sep 17 03:50:58.088591 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.088574    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"
Sep 17 03:50:58.090674 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.090649    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-0ccfd0\""
Sep 17 03:50:58.090989 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.090972    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-0ccfd0-dockercfg-7jjv9\""
Sep 17 03:50:58.100215 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.100187    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"]
Sep 17 03:50:58.125249 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.125208    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"]
Sep 17 03:50:58.128669 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.128643    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"
Sep 17 03:50:58.133736 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.133713    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gccm\" (UniqueName: \"kubernetes.io/projected/c60c67f8-9634-41f0-bf19-900e1ee80104-kube-api-access-8gccm\") pod \"payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw\" (UID: \"c60c67f8-9634-41f0-bf19-900e1ee80104\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"
Sep 17 03:50:58.133901 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.133875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c60c67f8-9634-41f0-bf19-900e1ee80104-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw\" (UID: \"c60c67f8-9634-41f0-bf19-900e1ee80104\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"
Sep 17 03:50:58.138144 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.138121    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"]
Sep 17 03:50:58.235317 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.235195    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l\" (UID: \"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"
Sep 17 03:50:58.235317 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.235307    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c60c67f8-9634-41f0-bf19-900e1ee80104-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw\" (UID: \"c60c67f8-9634-41f0-bf19-900e1ee80104\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"
Sep 17 03:50:58.235588 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.235350    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gccm\" (UniqueName: \"kubernetes.io/projected/c60c67f8-9634-41f0-bf19-900e1ee80104-kube-api-access-8gccm\") pod \"payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw\" (UID: \"c60c67f8-9634-41f0-bf19-900e1ee80104\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"
Sep 17 03:50:58.235588 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.235401    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9nzp\" (UniqueName: \"kubernetes.io/projected/8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e-kube-api-access-r9nzp\") pod \"payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l\" (UID: \"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"
Sep 17 03:50:58.236027 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.236006    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c60c67f8-9634-41f0-bf19-900e1ee80104-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw\" (UID: \"c60c67f8-9634-41f0-bf19-900e1ee80104\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"
Sep 17 03:50:58.242772 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.242751    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gccm\" (UniqueName: \"kubernetes.io/projected/c60c67f8-9634-41f0-bf19-900e1ee80104-kube-api-access-8gccm\") pod \"payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw\" (UID: \"c60c67f8-9634-41f0-bf19-900e1ee80104\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"
Sep 17 03:50:58.336289 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.336247    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l\" (UID: \"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"
Sep 17 03:50:58.336514 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.336332    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9nzp\" (UniqueName: \"kubernetes.io/projected/8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e-kube-api-access-r9nzp\") pod \"payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l\" (UID: \"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"
Sep 17 03:50:58.336985 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.336963    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l\" (UID: \"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"
Sep 17 03:50:58.344539 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.344513    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9nzp\" (UniqueName: \"kubernetes.io/projected/8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e-kube-api-access-r9nzp\") pod \"payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l\" (UID: \"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"
Sep 17 03:50:58.399991 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.399946    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"
Sep 17 03:50:58.439730 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.439692    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"
Sep 17 03:50:58.672095 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.672067    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"]
Sep 17 03:50:58.673584 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:50:58.673555    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc60c67f8_9634_41f0_bf19_900e1ee80104.slice/crio-f4298681e4b87746d1dee316b7685a1c7ef3b8ab6e051f2b9ddd98616bc581a3 WatchSource:0}: Error finding container f4298681e4b87746d1dee316b7685a1c7ef3b8ab6e051f2b9ddd98616bc581a3: Status 404 returned error can't find the container with id f4298681e4b87746d1dee316b7685a1c7ef3b8ab6e051f2b9ddd98616bc581a3
Sep 17 03:50:58.686832 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.686796    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw" event={"ID":"c60c67f8-9634-41f0-bf19-900e1ee80104","Type":"ContainerStarted","Data":"f4298681e4b87746d1dee316b7685a1c7ef3b8ab6e051f2b9ddd98616bc581a3"}
Sep 17 03:50:58.723333 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:58.723299    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"]
Sep 17 03:50:58.724620 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:50:58.724586    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f0e35ce_04c3_49ed_8793_f4eb6eff4c5e.slice/crio-6855e937d7d0d3b16805adccaf9e903815450109c18596d685c581c2bea1f008 WatchSource:0}: Error finding container 6855e937d7d0d3b16805adccaf9e903815450109c18596d685c581c2bea1f008: Status 404 returned error can't find the container with id 6855e937d7d0d3b16805adccaf9e903815450109c18596d685c581c2bea1f008
Sep 17 03:50:59.698494 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:59.698456    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l" event={"ID":"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e","Type":"ContainerStarted","Data":"54bb79a776e223fcde267bbc98fad91e9d6f72cdce1dcf6d72bb89eda4cdfc98"}
Sep 17 03:50:59.699001 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:59.698527    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l" event={"ID":"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e","Type":"ContainerStarted","Data":"6855e937d7d0d3b16805adccaf9e903815450109c18596d685c581c2bea1f008"}
Sep 17 03:50:59.699001 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:59.698597    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"
Sep 17 03:50:59.700416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:59.700388    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw" event={"ID":"c60c67f8-9634-41f0-bf19-900e1ee80104","Type":"ContainerStarted","Data":"19ebb5beb31613d6acca857d30911d964c550b7196defc2fbb9f2b9bab4dcba8"}
Sep 17 03:50:59.700552 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:59.700480    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"
Sep 17 03:50:59.721650 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:59.721589    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l" podStartSLOduration=1.7215696230000002 podStartE2EDuration="1.721569623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:50:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:50:59.717035759 +0000 UTC m=+1888.061761837" watchObservedRunningTime="2026-09-17 03:50:59.721569623 +0000 UTC m=+1888.066295706"
Sep 17 03:50:59.734681 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:50:59.734627    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw" podStartSLOduration=1.734610588 podStartE2EDuration="1.734610588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:50:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:50:59.733038008 +0000 UTC m=+1888.077764087" watchObservedRunningTime="2026-09-17 03:50:59.734610588 +0000 UTC m=+1888.079336666"
Sep 17 03:51:04.030827 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:04.030790    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:51:10.707765 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:10.707731    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"
Sep 17 03:51:10.708211 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:10.708091    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"
Sep 17 03:51:19.266964 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:19.266923    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:19.472636 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:19.472594    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:20.283511 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:20.283480    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:21.095193 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:21.095163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:21.798001 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:21.797968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:22.419919 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:22.419885    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:23.197555 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:23.197519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:23.840670 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:23.840644    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:24.484403 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:24.484373    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:25.523518 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:25.523483    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:26.000814 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.000782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:26.029502 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.029464    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"]
Sep 17 03:51:26.029739 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.029714    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l" podUID="8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e" containerName="payload-processing" containerID="cri-o://54bb79a776e223fcde267bbc98fad91e9d6f72cdce1dcf6d72bb89eda4cdfc98" gracePeriod=30
Sep 17 03:51:26.033809 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.033763    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"]
Sep 17 03:51:26.034133 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.034094    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw" podUID="c60c67f8-9634-41f0-bf19-900e1ee80104" containerName="payload-pre-processing" containerID="cri-o://19ebb5beb31613d6acca857d30911d964c550b7196defc2fbb9f2b9bab4dcba8" gracePeriod=30
Sep 17 03:51:26.103832 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.103806    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:26.196586 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.196550    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:26.253108 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.253026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:26.315833 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.315802    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"
Sep 17 03:51:26.319074 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.319053    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"
Sep 17 03:51:26.490979 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.490944    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e-plugins-config-volume\") pod \"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e\" (UID: \"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e\") "
Sep 17 03:51:26.491172 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.491027    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c60c67f8-9634-41f0-bf19-900e1ee80104-plugins-config-volume\") pod \"c60c67f8-9634-41f0-bf19-900e1ee80104\" (UID: \"c60c67f8-9634-41f0-bf19-900e1ee80104\") "
Sep 17 03:51:26.491172 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.491075    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gccm\" (UniqueName: \"kubernetes.io/projected/c60c67f8-9634-41f0-bf19-900e1ee80104-kube-api-access-8gccm\") pod \"c60c67f8-9634-41f0-bf19-900e1ee80104\" (UID: \"c60c67f8-9634-41f0-bf19-900e1ee80104\") "
Sep 17 03:51:26.491172 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.491118    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9nzp\" (UniqueName: \"kubernetes.io/projected/8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e-kube-api-access-r9nzp\") pod \"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e\" (UID: \"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e\") "
Sep 17 03:51:26.491396 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.491367    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e" (UID: "8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:51:26.491502 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.491480    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c60c67f8-9634-41f0-bf19-900e1ee80104-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c60c67f8-9634-41f0-bf19-900e1ee80104" (UID: "c60c67f8-9634-41f0-bf19-900e1ee80104"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:51:26.493327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.493292    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e-kube-api-access-r9nzp" (OuterVolumeSpecName: "kube-api-access-r9nzp") pod "8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e" (UID: "8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e"). InnerVolumeSpecName "kube-api-access-r9nzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:51:26.493327 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.493300    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c60c67f8-9634-41f0-bf19-900e1ee80104-kube-api-access-8gccm" (OuterVolumeSpecName: "kube-api-access-8gccm") pod "c60c67f8-9634-41f0-bf19-900e1ee80104" (UID: "c60c67f8-9634-41f0-bf19-900e1ee80104"). InnerVolumeSpecName "kube-api-access-8gccm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:51:26.569580 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.569547    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:26.592455 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.592392    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9nzp\" (UniqueName: \"kubernetes.io/projected/8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e-kube-api-access-r9nzp\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:51:26.592455 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.592424    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:51:26.592455 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.592460    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c60c67f8-9634-41f0-bf19-900e1ee80104-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:51:26.592687 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.592475    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gccm\" (UniqueName: \"kubernetes.io/projected/c60c67f8-9634-41f0-bf19-900e1ee80104-kube-api-access-8gccm\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:51:26.671079 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.671018    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:26.851309 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.851227    2576 generic.go:356] "Generic (PLEG): container finished" podID="8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e" containerID="54bb79a776e223fcde267bbc98fad91e9d6f72cdce1dcf6d72bb89eda4cdfc98" exitCode=0
Sep 17 03:51:26.851309 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.851272    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"
Sep 17 03:51:26.851537 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.851315    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l" event={"ID":"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e","Type":"ContainerDied","Data":"54bb79a776e223fcde267bbc98fad91e9d6f72cdce1dcf6d72bb89eda4cdfc98"}
Sep 17 03:51:26.851537 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.851366    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l" event={"ID":"8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e","Type":"ContainerDied","Data":"6855e937d7d0d3b16805adccaf9e903815450109c18596d685c581c2bea1f008"}
Sep 17 03:51:26.851537 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.851396    2576 scope.go:117] "RemoveContainer" containerID="54bb79a776e223fcde267bbc98fad91e9d6f72cdce1dcf6d72bb89eda4cdfc98"
Sep 17 03:51:26.852710 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.852690    2576 generic.go:356] "Generic (PLEG): container finished" podID="c60c67f8-9634-41f0-bf19-900e1ee80104" containerID="19ebb5beb31613d6acca857d30911d964c550b7196defc2fbb9f2b9bab4dcba8" exitCode=0
Sep 17 03:51:26.852825 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.852743    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw" event={"ID":"c60c67f8-9634-41f0-bf19-900e1ee80104","Type":"ContainerDied","Data":"19ebb5beb31613d6acca857d30911d964c550b7196defc2fbb9f2b9bab4dcba8"}
Sep 17 03:51:26.852825 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.852784    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw" event={"ID":"c60c67f8-9634-41f0-bf19-900e1ee80104","Type":"ContainerDied","Data":"f4298681e4b87746d1dee316b7685a1c7ef3b8ab6e051f2b9ddd98616bc581a3"}
Sep 17 03:51:26.852825 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.852792    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"
Sep 17 03:51:26.860579 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.860560    2576 scope.go:117] "RemoveContainer" containerID="54bb79a776e223fcde267bbc98fad91e9d6f72cdce1dcf6d72bb89eda4cdfc98"
Sep 17 03:51:26.860915 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:51:26.860882    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54bb79a776e223fcde267bbc98fad91e9d6f72cdce1dcf6d72bb89eda4cdfc98\": container with ID starting with 54bb79a776e223fcde267bbc98fad91e9d6f72cdce1dcf6d72bb89eda4cdfc98 not found: ID does not exist" containerID="54bb79a776e223fcde267bbc98fad91e9d6f72cdce1dcf6d72bb89eda4cdfc98"
Sep 17 03:51:26.861015 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.860925    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54bb79a776e223fcde267bbc98fad91e9d6f72cdce1dcf6d72bb89eda4cdfc98"} err="failed to get container status \"54bb79a776e223fcde267bbc98fad91e9d6f72cdce1dcf6d72bb89eda4cdfc98\": rpc error: code = NotFound desc = could not find container \"54bb79a776e223fcde267bbc98fad91e9d6f72cdce1dcf6d72bb89eda4cdfc98\": container with ID starting with 54bb79a776e223fcde267bbc98fad91e9d6f72cdce1dcf6d72bb89eda4cdfc98 not found: ID does not exist"
Sep 17 03:51:26.861015 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.860949    2576 scope.go:117] "RemoveContainer" containerID="19ebb5beb31613d6acca857d30911d964c550b7196defc2fbb9f2b9bab4dcba8"
Sep 17 03:51:26.870107 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.870069    2576 scope.go:117] "RemoveContainer" containerID="19ebb5beb31613d6acca857d30911d964c550b7196defc2fbb9f2b9bab4dcba8"
Sep 17 03:51:26.870416 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:51:26.870397    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19ebb5beb31613d6acca857d30911d964c550b7196defc2fbb9f2b9bab4dcba8\": container with ID starting with 19ebb5beb31613d6acca857d30911d964c550b7196defc2fbb9f2b9bab4dcba8 not found: ID does not exist" containerID="19ebb5beb31613d6acca857d30911d964c550b7196defc2fbb9f2b9bab4dcba8"
Sep 17 03:51:26.870505 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.870439    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19ebb5beb31613d6acca857d30911d964c550b7196defc2fbb9f2b9bab4dcba8"} err="failed to get container status \"19ebb5beb31613d6acca857d30911d964c550b7196defc2fbb9f2b9bab4dcba8\": rpc error: code = NotFound desc = could not find container \"19ebb5beb31613d6acca857d30911d964c550b7196defc2fbb9f2b9bab4dcba8\": container with ID starting with 19ebb5beb31613d6acca857d30911d964c550b7196defc2fbb9f2b9bab4dcba8 not found: ID does not exist"
Sep 17 03:51:26.877134 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.877113    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"]
Sep 17 03:51:26.879255 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.879232    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-0ccfd0-85967f5755-lff7l"]
Sep 17 03:51:26.887703 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.887616    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"]
Sep 17 03:51:26.889656 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:26.889636    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-0ccfd0-5d6bfc6fcd-vpjkw"]
Sep 17 03:51:27.310964 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:27.310927    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:28.113360 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:28.113328    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:28.294948 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:28.294914    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e" path="/var/lib/kubelet/pods/8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e/volumes"
Sep 17 03:51:28.295298 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:28.295285    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c60c67f8-9634-41f0-bf19-900e1ee80104" path="/var/lib/kubelet/pods/c60c67f8-9634-41f0-bf19-900e1ee80104/volumes"
Sep 17 03:51:28.743845 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:28.743811    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:29.495314 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:29.495281    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:30.827205 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:30.827175    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:51:32.286105 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:32.286073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:32.358090 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:32.358053    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:32.553734 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:32.553702    2576 scope.go:117] "RemoveContainer" containerID="1300bce36fd8f7731b93f7c8a0a2f3e7365654ded1b0713cdb124a5c269083dd"
Sep 17 03:51:32.560585 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:32.560544    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:36.217021 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:36.216986    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:36.652476 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:36.652421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:37.511329 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:37.511295    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:38.241289 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:38.241235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:38.977049 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:38.977011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:39.734535 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:39.734505    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:40.362570 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:40.362530    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:49.625558 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:49.625519    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:51:52.005160 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.005117    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"]
Sep 17 03:51:52.005800 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.005354    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7" podUID="15d03a6f-2ea1-4e27-9274-f45170ad095c" containerName="maas-api" containerID="cri-o://85fde3e927d9b90062294f4c16b6141a0830cdadc49ab423df5b2872e1940c21" gracePeriod=30
Sep 17 03:51:52.036366 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.036337    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:52.062956 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.062915    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"]
Sep 17 03:51:52.063225 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.063195    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5" podUID="1f775d34-3f35-492e-8a3f-ed05684ac8cd" containerName="payload-processing" containerID="cri-o://f5139fe5bf033e7e9409177430e03bf29ddd8bc414e68cf9f7a2137fe7f8ceb9" gracePeriod=30
Sep 17 03:51:52.136403 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.136361    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:52.222334 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.222299    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:52.281765 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.281643    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:52.319966 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.319936    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:51:52.345756 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.345723    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"
Sep 17 03:51:52.408539 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.408505    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l67tq\" (UniqueName: \"kubernetes.io/projected/1f775d34-3f35-492e-8a3f-ed05684ac8cd-kube-api-access-l67tq\") pod \"1f775d34-3f35-492e-8a3f-ed05684ac8cd\" (UID: \"1f775d34-3f35-492e-8a3f-ed05684ac8cd\") "
Sep 17 03:51:52.408753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.408565    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15d03a6f-2ea1-4e27-9274-f45170ad095c-metrics-certs\") pod \"15d03a6f-2ea1-4e27-9274-f45170ad095c\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") "
Sep 17 03:51:52.408753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.408585    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15d03a6f-2ea1-4e27-9274-f45170ad095c-openshift-service-ca\") pod \"15d03a6f-2ea1-4e27-9274-f45170ad095c\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") "
Sep 17 03:51:52.408753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.408609    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tlq5\" (UniqueName: \"kubernetes.io/projected/15d03a6f-2ea1-4e27-9274-f45170ad095c-kube-api-access-9tlq5\") pod \"15d03a6f-2ea1-4e27-9274-f45170ad095c\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") "
Sep 17 03:51:52.408753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.408628    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f775d34-3f35-492e-8a3f-ed05684ac8cd-plugins-config-volume\") pod \"1f775d34-3f35-492e-8a3f-ed05684ac8cd\" (UID: \"1f775d34-3f35-492e-8a3f-ed05684ac8cd\") "
Sep 17 03:51:52.408753 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.408717    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15d03a6f-2ea1-4e27-9274-f45170ad095c-maas-api-tls\") pod \"15d03a6f-2ea1-4e27-9274-f45170ad095c\" (UID: \"15d03a6f-2ea1-4e27-9274-f45170ad095c\") "
Sep 17 03:51:52.409042 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.408899    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15d03a6f-2ea1-4e27-9274-f45170ad095c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "15d03a6f-2ea1-4e27-9274-f45170ad095c" (UID: "15d03a6f-2ea1-4e27-9274-f45170ad095c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:51:52.409167 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.409146    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15d03a6f-2ea1-4e27-9274-f45170ad095c-openshift-service-ca\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:51:52.409232 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.409149    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f775d34-3f35-492e-8a3f-ed05684ac8cd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1f775d34-3f35-492e-8a3f-ed05684ac8cd" (UID: "1f775d34-3f35-492e-8a3f-ed05684ac8cd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:51:52.410867 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.410843    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15d03a6f-2ea1-4e27-9274-f45170ad095c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "15d03a6f-2ea1-4e27-9274-f45170ad095c" (UID: "15d03a6f-2ea1-4e27-9274-f45170ad095c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:51:52.410929 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.410850    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d03a6f-2ea1-4e27-9274-f45170ad095c-kube-api-access-9tlq5" (OuterVolumeSpecName: "kube-api-access-9tlq5") pod "15d03a6f-2ea1-4e27-9274-f45170ad095c" (UID: "15d03a6f-2ea1-4e27-9274-f45170ad095c"). InnerVolumeSpecName "kube-api-access-9tlq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:51:52.411144 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.411121    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15d03a6f-2ea1-4e27-9274-f45170ad095c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "15d03a6f-2ea1-4e27-9274-f45170ad095c" (UID: "15d03a6f-2ea1-4e27-9274-f45170ad095c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:51:52.411212 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.411129    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f775d34-3f35-492e-8a3f-ed05684ac8cd-kube-api-access-l67tq" (OuterVolumeSpecName: "kube-api-access-l67tq") pod "1f775d34-3f35-492e-8a3f-ed05684ac8cd" (UID: "1f775d34-3f35-492e-8a3f-ed05684ac8cd"). InnerVolumeSpecName "kube-api-access-l67tq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:51:52.509781 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.509747    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15d03a6f-2ea1-4e27-9274-f45170ad095c-metrics-certs\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:51:52.509781 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.509778    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tlq5\" (UniqueName: \"kubernetes.io/projected/15d03a6f-2ea1-4e27-9274-f45170ad095c-kube-api-access-9tlq5\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:51:52.509781 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.509789    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f775d34-3f35-492e-8a3f-ed05684ac8cd-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:51:52.510046 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.509798    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15d03a6f-2ea1-4e27-9274-f45170ad095c-maas-api-tls\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:51:52.510046 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.509808    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l67tq\" (UniqueName: \"kubernetes.io/projected/1f775d34-3f35-492e-8a3f-ed05684ac8cd-kube-api-access-l67tq\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:51:52.620513 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.620464    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:52.720843 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.720814    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:52.997824 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.997740    2576 generic.go:356] "Generic (PLEG): container finished" podID="15d03a6f-2ea1-4e27-9274-f45170ad095c" containerID="85fde3e927d9b90062294f4c16b6141a0830cdadc49ab423df5b2872e1940c21" exitCode=0
Sep 17 03:51:52.997824 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.997799    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"
Sep 17 03:51:52.998037 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.997828    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7" event={"ID":"15d03a6f-2ea1-4e27-9274-f45170ad095c","Type":"ContainerDied","Data":"85fde3e927d9b90062294f4c16b6141a0830cdadc49ab423df5b2872e1940c21"}
Sep 17 03:51:52.998037 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.997908    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7" event={"ID":"15d03a6f-2ea1-4e27-9274-f45170ad095c","Type":"ContainerDied","Data":"c4df0c8401a9869bf7dbdfb900493bceacd8b8fed792e1ee5d09a21da6595b78"}
Sep 17 03:51:52.998037 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.997943    2576 scope.go:117] "RemoveContainer" containerID="85fde3e927d9b90062294f4c16b6141a0830cdadc49ab423df5b2872e1940c21"
Sep 17 03:51:52.999285 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.999227    2576 generic.go:356] "Generic (PLEG): container finished" podID="1f775d34-3f35-492e-8a3f-ed05684ac8cd" containerID="f5139fe5bf033e7e9409177430e03bf29ddd8bc414e68cf9f7a2137fe7f8ceb9" exitCode=0
Sep 17 03:51:52.999285 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.999254    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5" event={"ID":"1f775d34-3f35-492e-8a3f-ed05684ac8cd","Type":"ContainerDied","Data":"f5139fe5bf033e7e9409177430e03bf29ddd8bc414e68cf9f7a2137fe7f8ceb9"}
Sep 17 03:51:52.999389 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.999296    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5" event={"ID":"1f775d34-3f35-492e-8a3f-ed05684ac8cd","Type":"ContainerDied","Data":"822fb25226f975050faca247871efde69ef76da9611d5cfc6dc99414895c7fd5"}
Sep 17 03:51:52.999389 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:52.999294    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"
Sep 17 03:51:53.008773 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:53.008615    2576 scope.go:117] "RemoveContainer" containerID="85fde3e927d9b90062294f4c16b6141a0830cdadc49ab423df5b2872e1940c21"
Sep 17 03:51:53.009014 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:51:53.008998    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85fde3e927d9b90062294f4c16b6141a0830cdadc49ab423df5b2872e1940c21\": container with ID starting with 85fde3e927d9b90062294f4c16b6141a0830cdadc49ab423df5b2872e1940c21 not found: ID does not exist" containerID="85fde3e927d9b90062294f4c16b6141a0830cdadc49ab423df5b2872e1940c21"
Sep 17 03:51:53.009057 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:53.009024    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85fde3e927d9b90062294f4c16b6141a0830cdadc49ab423df5b2872e1940c21"} err="failed to get container status \"85fde3e927d9b90062294f4c16b6141a0830cdadc49ab423df5b2872e1940c21\": rpc error: code = NotFound desc = could not find container \"85fde3e927d9b90062294f4c16b6141a0830cdadc49ab423df5b2872e1940c21\": container with ID starting with 85fde3e927d9b90062294f4c16b6141a0830cdadc49ab423df5b2872e1940c21 not found: ID does not exist"
Sep 17 03:51:53.009057 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:53.009041    2576 scope.go:117] "RemoveContainer" containerID="f5139fe5bf033e7e9409177430e03bf29ddd8bc414e68cf9f7a2137fe7f8ceb9"
Sep 17 03:51:53.019486 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:53.019467    2576 scope.go:117] "RemoveContainer" containerID="f5139fe5bf033e7e9409177430e03bf29ddd8bc414e68cf9f7a2137fe7f8ceb9"
Sep 17 03:51:53.020038 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:51:53.019982    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5139fe5bf033e7e9409177430e03bf29ddd8bc414e68cf9f7a2137fe7f8ceb9\": container with ID starting with f5139fe5bf033e7e9409177430e03bf29ddd8bc414e68cf9f7a2137fe7f8ceb9 not found: ID does not exist" containerID="f5139fe5bf033e7e9409177430e03bf29ddd8bc414e68cf9f7a2137fe7f8ceb9"
Sep 17 03:51:53.020185 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:53.020047    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5139fe5bf033e7e9409177430e03bf29ddd8bc414e68cf9f7a2137fe7f8ceb9"} err="failed to get container status \"f5139fe5bf033e7e9409177430e03bf29ddd8bc414e68cf9f7a2137fe7f8ceb9\": rpc error: code = NotFound desc = could not find container \"f5139fe5bf033e7e9409177430e03bf29ddd8bc414e68cf9f7a2137fe7f8ceb9\": container with ID starting with f5139fe5bf033e7e9409177430e03bf29ddd8bc414e68cf9f7a2137fe7f8ceb9 not found: ID does not exist"
Sep 17 03:51:53.023465 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:53.023416    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"]
Sep 17 03:51:53.026067 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:53.026045    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-d0ab1a-7cf7c55465-klds7"]
Sep 17 03:51:53.033830 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:53.033800    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"]
Sep 17 03:51:53.037866 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:53.037845    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d0ab1a-6f69c6df59-knxq5"]
Sep 17 03:51:53.285754 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:53.285661    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:54.099005 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:54.098975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:54.294047 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:54.294012    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15d03a6f-2ea1-4e27-9274-f45170ad095c" path="/var/lib/kubelet/pods/15d03a6f-2ea1-4e27-9274-f45170ad095c/volumes"
Sep 17 03:51:54.294379 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:54.294366    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f775d34-3f35-492e-8a3f-ed05684ac8cd" path="/var/lib/kubelet/pods/1f775d34-3f35-492e-8a3f-ed05684ac8cd/volumes"
Sep 17 03:51:54.762106 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:54.762076    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:51:57.705265 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:51:57.705227    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:52:02.631117 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:52:02.631079    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:52:35.127026 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:52:35.126990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:52:35.228312 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:52:35.228275    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:52:40.759738 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:52:40.759704    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:03.446881 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:03.446847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:03.547858 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:03.547826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:03.721878 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:03.721793    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:04.057721 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.057688    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct"]
Sep 17 03:53:04.058069 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.058057    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d03a6f-2ea1-4e27-9274-f45170ad095c" containerName="maas-api"
Sep 17 03:53:04.058114 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.058071    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d03a6f-2ea1-4e27-9274-f45170ad095c" containerName="maas-api"
Sep 17 03:53:04.058114 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.058081    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c60c67f8-9634-41f0-bf19-900e1ee80104" containerName="payload-pre-processing"
Sep 17 03:53:04.058114 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.058088    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c60c67f8-9634-41f0-bf19-900e1ee80104" containerName="payload-pre-processing"
Sep 17 03:53:04.058114 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.058096    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e" containerName="payload-processing"
Sep 17 03:53:04.058114 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.058101    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e" containerName="payload-processing"
Sep 17 03:53:04.058114 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.058113    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f775d34-3f35-492e-8a3f-ed05684ac8cd" containerName="payload-processing"
Sep 17 03:53:04.058303 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.058119    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f775d34-3f35-492e-8a3f-ed05684ac8cd" containerName="payload-processing"
Sep 17 03:53:04.058303 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.058170    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0e35ce-04c3-49ed-8793-f4eb6eff4c5e" containerName="payload-processing"
Sep 17 03:53:04.058303 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.058180    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f775d34-3f35-492e-8a3f-ed05684ac8cd" containerName="payload-processing"
Sep 17 03:53:04.058303 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.058186    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c60c67f8-9634-41f0-bf19-900e1ee80104" containerName="payload-pre-processing"
Sep 17 03:53:04.058303 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.058192    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d03a6f-2ea1-4e27-9274-f45170ad095c" containerName="maas-api"
Sep 17 03:53:04.061380 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.061363    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct"
Sep 17 03:53:04.063345 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.063324    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8vwnw\""
Sep 17 03:53:04.068445 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.068401    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct"]
Sep 17 03:53:04.145949 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.145912    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07992b71-afb2-4202-8116-c33b06dd5c98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct\" (UID: \"07992b71-afb2-4202-8116-c33b06dd5c98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct"
Sep 17 03:53:04.246700 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.246664    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07992b71-afb2-4202-8116-c33b06dd5c98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct\" (UID: \"07992b71-afb2-4202-8116-c33b06dd5c98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct"
Sep 17 03:53:04.247306 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.247283    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07992b71-afb2-4202-8116-c33b06dd5c98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct\" (UID: \"07992b71-afb2-4202-8116-c33b06dd5c98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct"
Sep 17 03:53:04.374236 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.374139    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct"
Sep 17 03:53:04.410344 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.410310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:04.645411 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:04.645379    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct"]
Sep 17 03:53:04.646867 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:53:04.646828    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07992b71_afb2_4202_8116_c33b06dd5c98.slice/crio-d1b2ef34c0d4c7de460aa61fc9f3c564f3237630f1810266067ed129080e7a62 WatchSource:0}: Error finding container d1b2ef34c0d4c7de460aa61fc9f3c564f3237630f1810266067ed129080e7a62: Status 404 returned error can't find the container with id d1b2ef34c0d4c7de460aa61fc9f3c564f3237630f1810266067ed129080e7a62
Sep 17 03:53:05.303359 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:05.303325    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:05.402143 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:05.402112    2576 generic.go:356] "Generic (PLEG): container finished" podID="07992b71-afb2-4202-8116-c33b06dd5c98" containerID="5b869ae825ed3d7d27c486926a662da8a76f4038bbee43afbc76c09be11b8fac" exitCode=0
Sep 17 03:53:05.402316 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:05.402192    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct" event={"ID":"07992b71-afb2-4202-8116-c33b06dd5c98","Type":"ContainerDied","Data":"5b869ae825ed3d7d27c486926a662da8a76f4038bbee43afbc76c09be11b8fac"}
Sep 17 03:53:05.402316 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:05.402242    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct" event={"ID":"07992b71-afb2-4202-8116-c33b06dd5c98","Type":"ContainerStarted","Data":"d1b2ef34c0d4c7de460aa61fc9f3c564f3237630f1810266067ed129080e7a62"}
Sep 17 03:53:05.953017 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:05.952983    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:06.536585 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:06.536562    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct"
Sep 17 03:53:06.668572 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:06.668483    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07992b71-afb2-4202-8116-c33b06dd5c98-maas-api-service-ca\") pod \"07992b71-afb2-4202-8116-c33b06dd5c98\" (UID: \"07992b71-afb2-4202-8116-c33b06dd5c98\") "
Sep 17 03:53:06.668844 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:06.668823    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07992b71-afb2-4202-8116-c33b06dd5c98-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "07992b71-afb2-4202-8116-c33b06dd5c98" (UID: "07992b71-afb2-4202-8116-c33b06dd5c98"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:53:06.689005 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:06.688971    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:06.769890 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:06.769857    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07992b71-afb2-4202-8116-c33b06dd5c98-maas-api-service-ca\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:53:07.412761 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:07.412727    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct" event={"ID":"07992b71-afb2-4202-8116-c33b06dd5c98","Type":"ContainerDied","Data":"d1b2ef34c0d4c7de460aa61fc9f3c564f3237630f1810266067ed129080e7a62"}
Sep 17 03:53:07.412761 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:07.412765    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct"
Sep 17 03:53:07.413203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:07.412775    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1b2ef34c0d4c7de460aa61fc9f3c564f3237630f1810266067ed129080e7a62"
Sep 17 03:53:07.414936 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:07.414906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:14.482034 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.481999    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:14.532667 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.532627    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"]
Sep 17 03:53:14.533010 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.532956    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t" podUID="23cee20c-e748-4681-872f-1b6c2bba21b5" containerName="payload-pre-processing" containerID="cri-o://70a1d56a77787d160e0ca1776b45e151c181f10f887825650ad985df50afea51" gracePeriod=30
Sep 17 03:53:14.553386 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.553305    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:14.614729 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.614691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:14.676612 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.676573    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct"]
Sep 17 03:53:14.683055 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.683018    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-84cd5d-v2wct"]
Sep 17 03:53:14.684999 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.684972    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:14.747598 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.747515    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:14.817876 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.817849    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"
Sep 17 03:53:14.943228 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.943189    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23cee20c-e748-4681-872f-1b6c2bba21b5-plugins-config-volume\") pod \"23cee20c-e748-4681-872f-1b6c2bba21b5\" (UID: \"23cee20c-e748-4681-872f-1b6c2bba21b5\") "
Sep 17 03:53:14.943465 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.943273    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58x2p\" (UniqueName: \"kubernetes.io/projected/23cee20c-e748-4681-872f-1b6c2bba21b5-kube-api-access-58x2p\") pod \"23cee20c-e748-4681-872f-1b6c2bba21b5\" (UID: \"23cee20c-e748-4681-872f-1b6c2bba21b5\") "
Sep 17 03:53:14.943625 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.943598    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23cee20c-e748-4681-872f-1b6c2bba21b5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "23cee20c-e748-4681-872f-1b6c2bba21b5" (UID: "23cee20c-e748-4681-872f-1b6c2bba21b5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:53:14.945469 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:14.945443    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23cee20c-e748-4681-872f-1b6c2bba21b5-kube-api-access-58x2p" (OuterVolumeSpecName: "kube-api-access-58x2p") pod "23cee20c-e748-4681-872f-1b6c2bba21b5" (UID: "23cee20c-e748-4681-872f-1b6c2bba21b5"). InnerVolumeSpecName "kube-api-access-58x2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:53:15.044095 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.043990    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23cee20c-e748-4681-872f-1b6c2bba21b5-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:53:15.044095 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.044021    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58x2p\" (UniqueName: \"kubernetes.io/projected/23cee20c-e748-4681-872f-1b6c2bba21b5-kube-api-access-58x2p\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:53:15.044494 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.044460    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:15.146878 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.146843    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:15.457662 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.457631    2576 generic.go:356] "Generic (PLEG): container finished" podID="23cee20c-e748-4681-872f-1b6c2bba21b5" containerID="70a1d56a77787d160e0ca1776b45e151c181f10f887825650ad985df50afea51" exitCode=0
Sep 17 03:53:15.457866 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.457678    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"
Sep 17 03:53:15.457866 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.457724    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t" event={"ID":"23cee20c-e748-4681-872f-1b6c2bba21b5","Type":"ContainerDied","Data":"70a1d56a77787d160e0ca1776b45e151c181f10f887825650ad985df50afea51"}
Sep 17 03:53:15.457866 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.457768    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t" event={"ID":"23cee20c-e748-4681-872f-1b6c2bba21b5","Type":"ContainerDied","Data":"5a18f06579e1ae1a80214adf64fb5d6c3b8fcfc910e69496c7e9f2a834461342"}
Sep 17 03:53:15.457866 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.457793    2576 scope.go:117] "RemoveContainer" containerID="70a1d56a77787d160e0ca1776b45e151c181f10f887825650ad985df50afea51"
Sep 17 03:53:15.466984 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.466962    2576 scope.go:117] "RemoveContainer" containerID="70a1d56a77787d160e0ca1776b45e151c181f10f887825650ad985df50afea51"
Sep 17 03:53:15.467250 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:53:15.467229    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70a1d56a77787d160e0ca1776b45e151c181f10f887825650ad985df50afea51\": container with ID starting with 70a1d56a77787d160e0ca1776b45e151c181f10f887825650ad985df50afea51 not found: ID does not exist" containerID="70a1d56a77787d160e0ca1776b45e151c181f10f887825650ad985df50afea51"
Sep 17 03:53:15.467293 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.467259    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70a1d56a77787d160e0ca1776b45e151c181f10f887825650ad985df50afea51"} err="failed to get container status \"70a1d56a77787d160e0ca1776b45e151c181f10f887825650ad985df50afea51\": rpc error: code = NotFound desc = could not find container \"70a1d56a77787d160e0ca1776b45e151c181f10f887825650ad985df50afea51\": container with ID starting with 70a1d56a77787d160e0ca1776b45e151c181f10f887825650ad985df50afea51 not found: ID does not exist"
Sep 17 03:53:15.481499 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.481468    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"]
Sep 17 03:53:15.484328 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.484300    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-84cd5d-8449cc9d5d-sfx5t"]
Sep 17 03:53:15.914968 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:15.914932    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:16.295290 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:16.295210    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07992b71-afb2-4202-8116-c33b06dd5c98" path="/var/lib/kubelet/pods/07992b71-afb2-4202-8116-c33b06dd5c98/volumes"
Sep 17 03:53:16.295613 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:16.295599    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23cee20c-e748-4681-872f-1b6c2bba21b5" path="/var/lib/kubelet/pods/23cee20c-e748-4681-872f-1b6c2bba21b5/volumes"
Sep 17 03:53:16.567910 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:16.567868    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:17.837635 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:17.837597    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:53:20.827837 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:20.827794    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:20.932690 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:20.932657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:21.117673 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.117592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:21.455970 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.455890    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77"]
Sep 17 03:53:21.456236 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.456225    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07992b71-afb2-4202-8116-c33b06dd5c98" containerName="revoke-keys"
Sep 17 03:53:21.456286 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.456238    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="07992b71-afb2-4202-8116-c33b06dd5c98" containerName="revoke-keys"
Sep 17 03:53:21.456286 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.456251    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23cee20c-e748-4681-872f-1b6c2bba21b5" containerName="payload-pre-processing"
Sep 17 03:53:21.456286 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.456256    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="23cee20c-e748-4681-872f-1b6c2bba21b5" containerName="payload-pre-processing"
Sep 17 03:53:21.456375 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.456323    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="23cee20c-e748-4681-872f-1b6c2bba21b5" containerName="payload-pre-processing"
Sep 17 03:53:21.456375 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.456332    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="07992b71-afb2-4202-8116-c33b06dd5c98" containerName="revoke-keys"
Sep 17 03:53:21.460939 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.460921    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77"
Sep 17 03:53:21.463571 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.463539    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8vwnw\""
Sep 17 03:53:21.466133 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.466108    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77"]
Sep 17 03:53:21.497793 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.497763    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d20faa63-e131-4865-b0a9-bbeabe709b35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77\" (UID: \"d20faa63-e131-4865-b0a9-bbeabe709b35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77"
Sep 17 03:53:21.598892 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.598849    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d20faa63-e131-4865-b0a9-bbeabe709b35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77\" (UID: \"d20faa63-e131-4865-b0a9-bbeabe709b35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77"
Sep 17 03:53:21.599573 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.599547    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d20faa63-e131-4865-b0a9-bbeabe709b35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77\" (UID: \"d20faa63-e131-4865-b0a9-bbeabe709b35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77"
Sep 17 03:53:21.773056 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.772960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77"
Sep 17 03:53:21.799380 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:21.799345    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:22.048473 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:22.048419    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77"]
Sep 17 03:53:22.049751 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:53:22.049721    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd20faa63_e131_4865_b0a9_bbeabe709b35.slice/crio-7a87dff8e0c93987fc512336c31b010d1fb11d33533b393f81dd72449a0a9902 WatchSource:0}: Error finding container 7a87dff8e0c93987fc512336c31b010d1fb11d33533b393f81dd72449a0a9902: Status 404 returned error can't find the container with id 7a87dff8e0c93987fc512336c31b010d1fb11d33533b393f81dd72449a0a9902
Sep 17 03:53:22.499317 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:22.499272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77" event={"ID":"d20faa63-e131-4865-b0a9-bbeabe709b35","Type":"ContainerStarted","Data":"47d8421167d66b9bff4a0dad19d03306a5e1849213d0509205a214d62344a93f"}
Sep 17 03:53:22.499514 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:22.499340    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77" event={"ID":"d20faa63-e131-4865-b0a9-bbeabe709b35","Type":"ContainerStarted","Data":"7a87dff8e0c93987fc512336c31b010d1fb11d33533b393f81dd72449a0a9902"}
Sep 17 03:53:22.515201 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:22.515145    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77" podStartSLOduration=1.515129795 podStartE2EDuration="1.515129795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:53:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:53:22.513325153 +0000 UTC m=+2030.858051224" watchObservedRunningTime="2026-09-17 03:53:22.515129795 +0000 UTC m=+2030.859855865"
Sep 17 03:53:22.762994 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:22.762961    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:23.429102 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:23.429071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:23.508417 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:23.508385    2576 generic.go:356] "Generic (PLEG): container finished" podID="d20faa63-e131-4865-b0a9-bbeabe709b35" containerID="47d8421167d66b9bff4a0dad19d03306a5e1849213d0509205a214d62344a93f" exitCode=0
Sep 17 03:53:23.508688 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:23.508478    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77" event={"ID":"d20faa63-e131-4865-b0a9-bbeabe709b35","Type":"ContainerDied","Data":"47d8421167d66b9bff4a0dad19d03306a5e1849213d0509205a214d62344a93f"}
Sep 17 03:53:24.185004 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:24.184967    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:24.650537 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:24.650507    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77"
Sep 17 03:53:24.728452 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:24.728395    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d20faa63-e131-4865-b0a9-bbeabe709b35-maas-api-service-ca\") pod \"d20faa63-e131-4865-b0a9-bbeabe709b35\" (UID: \"d20faa63-e131-4865-b0a9-bbeabe709b35\") "
Sep 17 03:53:24.728777 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:24.728753    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d20faa63-e131-4865-b0a9-bbeabe709b35-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d20faa63-e131-4865-b0a9-bbeabe709b35" (UID: "d20faa63-e131-4865-b0a9-bbeabe709b35"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:53:24.829278 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:24.829248    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d20faa63-e131-4865-b0a9-bbeabe709b35-maas-api-service-ca\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:53:25.521369 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:25.521337    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77"
Sep 17 03:53:25.521568 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:25.521360    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77" event={"ID":"d20faa63-e131-4865-b0a9-bbeabe709b35","Type":"ContainerDied","Data":"7a87dff8e0c93987fc512336c31b010d1fb11d33533b393f81dd72449a0a9902"}
Sep 17 03:53:25.521568 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:25.521418    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a87dff8e0c93987fc512336c31b010d1fb11d33533b393f81dd72449a0a9902"
Sep 17 03:53:31.885801 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:31.885768    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"]
Sep 17 03:53:31.886324 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:31.886292    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh" podUID="fc41ede3-20c0-40b8-a131-f8979f03a1e1" containerName="maas-api" containerID="cri-o://1293da3aaa0b7e209dcbab3cb07f88dbb972e37e87a26af4717e325d59f51ed3" gracePeriod=30
Sep 17 03:53:31.914566 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:31.914533    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:31.940135 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:31.940102    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"]
Sep 17 03:53:31.940387 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:31.940362    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7" podUID="941c982b-fa6f-4f6f-b088-589bf23e7162" containerName="payload-processing" containerID="cri-o://5ca225bf0332935212d5d3b652130cdf33abd050b4d7ade884e1fc1e5ae82512" gracePeriod=30
Sep 17 03:53:31.948248 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:31.948205    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"]
Sep 17 03:53:31.948500 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:31.948466    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv" podUID="3f7764ac-5295-4921-aa0c-2e132638125c" containerName="payload-pre-processing" containerID="cri-o://b64c11fb315bcfd42cb83c284b4bcaa758951274e917ab740b702d4b691d5a61" gracePeriod=30
Sep 17 03:53:31.989240 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:31.989196    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:32.046829 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.046797    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:32.100374 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.100328    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77"]
Sep 17 03:53:32.103721 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.103692    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-22e045-gdb77"]
Sep 17 03:53:32.106273 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.106240    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:32.163394 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.163344    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:32.213584 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.213556    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"
Sep 17 03:53:32.216358 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.216340    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:53:32.248194 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.247344    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"
Sep 17 03:53:32.293016 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.292982    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w6q8\" (UniqueName: \"kubernetes.io/projected/941c982b-fa6f-4f6f-b088-589bf23e7162-kube-api-access-7w6q8\") pod \"941c982b-fa6f-4f6f-b088-589bf23e7162\" (UID: \"941c982b-fa6f-4f6f-b088-589bf23e7162\") "
Sep 17 03:53:32.293200 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.293024    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-maas-api-tls\") pod \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") "
Sep 17 03:53:32.293200 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.293061    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhdj7\" (UniqueName: \"kubernetes.io/projected/fc41ede3-20c0-40b8-a131-f8979f03a1e1-kube-api-access-dhdj7\") pod \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") "
Sep 17 03:53:32.293200 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.293089    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc41ede3-20c0-40b8-a131-f8979f03a1e1-openshift-service-ca\") pod \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") "
Sep 17 03:53:32.293200 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.293118    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f7764ac-5295-4921-aa0c-2e132638125c-plugins-config-volume\") pod \"3f7764ac-5295-4921-aa0c-2e132638125c\" (UID: \"3f7764ac-5295-4921-aa0c-2e132638125c\") "
Sep 17 03:53:32.293200 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.293156    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77tsq\" (UniqueName: \"kubernetes.io/projected/3f7764ac-5295-4921-aa0c-2e132638125c-kube-api-access-77tsq\") pod \"3f7764ac-5295-4921-aa0c-2e132638125c\" (UID: \"3f7764ac-5295-4921-aa0c-2e132638125c\") "
Sep 17 03:53:32.293525 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.293500    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/941c982b-fa6f-4f6f-b088-589bf23e7162-plugins-config-volume\") pod \"941c982b-fa6f-4f6f-b088-589bf23e7162\" (UID: \"941c982b-fa6f-4f6f-b088-589bf23e7162\") "
Sep 17 03:53:32.293606 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.293581    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-metrics-certs\") pod \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\" (UID: \"fc41ede3-20c0-40b8-a131-f8979f03a1e1\") "
Sep 17 03:53:32.293668 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.293593    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc41ede3-20c0-40b8-a131-f8979f03a1e1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fc41ede3-20c0-40b8-a131-f8979f03a1e1" (UID: "fc41ede3-20c0-40b8-a131-f8979f03a1e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:53:32.295182 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.293853    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc41ede3-20c0-40b8-a131-f8979f03a1e1-openshift-service-ca\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:53:32.295182 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.293980    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/941c982b-fa6f-4f6f-b088-589bf23e7162-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "941c982b-fa6f-4f6f-b088-589bf23e7162" (UID: "941c982b-fa6f-4f6f-b088-589bf23e7162"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:53:32.295182 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.294278    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f7764ac-5295-4921-aa0c-2e132638125c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f7764ac-5295-4921-aa0c-2e132638125c" (UID: "3f7764ac-5295-4921-aa0c-2e132638125c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:53:32.295416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.295320    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/941c982b-fa6f-4f6f-b088-589bf23e7162-kube-api-access-7w6q8" (OuterVolumeSpecName: "kube-api-access-7w6q8") pod "941c982b-fa6f-4f6f-b088-589bf23e7162" (UID: "941c982b-fa6f-4f6f-b088-589bf23e7162"). InnerVolumeSpecName "kube-api-access-7w6q8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:53:32.295658 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.295628    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f7764ac-5295-4921-aa0c-2e132638125c-kube-api-access-77tsq" (OuterVolumeSpecName: "kube-api-access-77tsq") pod "3f7764ac-5295-4921-aa0c-2e132638125c" (UID: "3f7764ac-5295-4921-aa0c-2e132638125c"). InnerVolumeSpecName "kube-api-access-77tsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:53:32.295748 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.295672    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc41ede3-20c0-40b8-a131-f8979f03a1e1-kube-api-access-dhdj7" (OuterVolumeSpecName: "kube-api-access-dhdj7") pod "fc41ede3-20c0-40b8-a131-f8979f03a1e1" (UID: "fc41ede3-20c0-40b8-a131-f8979f03a1e1"). InnerVolumeSpecName "kube-api-access-dhdj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:53:32.295748 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.295708    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fc41ede3-20c0-40b8-a131-f8979f03a1e1" (UID: "fc41ede3-20c0-40b8-a131-f8979f03a1e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:53:32.295847 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.295827    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fc41ede3-20c0-40b8-a131-f8979f03a1e1" (UID: "fc41ede3-20c0-40b8-a131-f8979f03a1e1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 03:53:32.295916 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.295896    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d20faa63-e131-4865-b0a9-bbeabe709b35" path="/var/lib/kubelet/pods/d20faa63-e131-4865-b0a9-bbeabe709b35/volumes"
Sep 17 03:53:32.394673 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.394638    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w6q8\" (UniqueName: \"kubernetes.io/projected/941c982b-fa6f-4f6f-b088-589bf23e7162-kube-api-access-7w6q8\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:53:32.394673 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.394666    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-maas-api-tls\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:53:32.394673 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.394676    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhdj7\" (UniqueName: \"kubernetes.io/projected/fc41ede3-20c0-40b8-a131-f8979f03a1e1-kube-api-access-dhdj7\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:53:32.394905 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.394685    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f7764ac-5295-4921-aa0c-2e132638125c-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:53:32.394905 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.394694    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77tsq\" (UniqueName: \"kubernetes.io/projected/3f7764ac-5295-4921-aa0c-2e132638125c-kube-api-access-77tsq\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:53:32.394905 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.394703    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/941c982b-fa6f-4f6f-b088-589bf23e7162-plugins-config-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:53:32.394905 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.394711    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc41ede3-20c0-40b8-a131-f8979f03a1e1-metrics-certs\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:53:32.485629 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.485522    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:32.559415 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.559384    2576 generic.go:356] "Generic (PLEG): container finished" podID="941c982b-fa6f-4f6f-b088-589bf23e7162" containerID="5ca225bf0332935212d5d3b652130cdf33abd050b4d7ade884e1fc1e5ae82512" exitCode=0
Sep 17 03:53:32.559415 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.559439    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"
Sep 17 03:53:32.559682 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.559471    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7" event={"ID":"941c982b-fa6f-4f6f-b088-589bf23e7162","Type":"ContainerDied","Data":"5ca225bf0332935212d5d3b652130cdf33abd050b4d7ade884e1fc1e5ae82512"}
Sep 17 03:53:32.559682 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.559521    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7" event={"ID":"941c982b-fa6f-4f6f-b088-589bf23e7162","Type":"ContainerDied","Data":"0e7ee66987880bf00a016703a7b937351b8111f5cd1b362141a6c898e5827f06"}
Sep 17 03:53:32.559682 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.559553    2576 scope.go:117] "RemoveContainer" containerID="5ca225bf0332935212d5d3b652130cdf33abd050b4d7ade884e1fc1e5ae82512"
Sep 17 03:53:32.561103 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.561070    2576 generic.go:356] "Generic (PLEG): container finished" podID="3f7764ac-5295-4921-aa0c-2e132638125c" containerID="b64c11fb315bcfd42cb83c284b4bcaa758951274e917ab740b702d4b691d5a61" exitCode=0
Sep 17 03:53:32.561213 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.561135    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"
Sep 17 03:53:32.561213 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.561162    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv" event={"ID":"3f7764ac-5295-4921-aa0c-2e132638125c","Type":"ContainerDied","Data":"b64c11fb315bcfd42cb83c284b4bcaa758951274e917ab740b702d4b691d5a61"}
Sep 17 03:53:32.561213 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.561200    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv" event={"ID":"3f7764ac-5295-4921-aa0c-2e132638125c","Type":"ContainerDied","Data":"ba7041dd2bf463b36662a028d75eaa7300b58caa8caedf6b3bc413996bb346a7"}
Sep 17 03:53:32.563077 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.563035    2576 generic.go:356] "Generic (PLEG): container finished" podID="fc41ede3-20c0-40b8-a131-f8979f03a1e1" containerID="1293da3aaa0b7e209dcbab3cb07f88dbb972e37e87a26af4717e325d59f51ed3" exitCode=0
Sep 17 03:53:32.563203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.563067    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh" event={"ID":"fc41ede3-20c0-40b8-a131-f8979f03a1e1","Type":"ContainerDied","Data":"1293da3aaa0b7e209dcbab3cb07f88dbb972e37e87a26af4717e325d59f51ed3"}
Sep 17 03:53:32.563203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.563122    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh" event={"ID":"fc41ede3-20c0-40b8-a131-f8979f03a1e1","Type":"ContainerDied","Data":"aef50505a79301c20639273c9583b3c0718b9ae29ed8233510f0dfb3d386a874"}
Sep 17 03:53:32.563203 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.563122    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"
Sep 17 03:53:32.569647 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.569632    2576 scope.go:117] "RemoveContainer" containerID="5ca225bf0332935212d5d3b652130cdf33abd050b4d7ade884e1fc1e5ae82512"
Sep 17 03:53:32.569946 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:53:32.569927    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ca225bf0332935212d5d3b652130cdf33abd050b4d7ade884e1fc1e5ae82512\": container with ID starting with 5ca225bf0332935212d5d3b652130cdf33abd050b4d7ade884e1fc1e5ae82512 not found: ID does not exist" containerID="5ca225bf0332935212d5d3b652130cdf33abd050b4d7ade884e1fc1e5ae82512"
Sep 17 03:53:32.569994 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.569954    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ca225bf0332935212d5d3b652130cdf33abd050b4d7ade884e1fc1e5ae82512"} err="failed to get container status \"5ca225bf0332935212d5d3b652130cdf33abd050b4d7ade884e1fc1e5ae82512\": rpc error: code = NotFound desc = could not find container \"5ca225bf0332935212d5d3b652130cdf33abd050b4d7ade884e1fc1e5ae82512\": container with ID starting with 5ca225bf0332935212d5d3b652130cdf33abd050b4d7ade884e1fc1e5ae82512 not found: ID does not exist"
Sep 17 03:53:32.569994 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.569971    2576 scope.go:117] "RemoveContainer" containerID="b64c11fb315bcfd42cb83c284b4bcaa758951274e917ab740b702d4b691d5a61"
Sep 17 03:53:32.579632 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.579597    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"]
Sep 17 03:53:32.582127 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.582099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:32.582673 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.582654    2576 scope.go:117] "RemoveContainer" containerID="b64c11fb315bcfd42cb83c284b4bcaa758951274e917ab740b702d4b691d5a61"
Sep 17 03:53:32.583009 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:53:32.582984    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b64c11fb315bcfd42cb83c284b4bcaa758951274e917ab740b702d4b691d5a61\": container with ID starting with b64c11fb315bcfd42cb83c284b4bcaa758951274e917ab740b702d4b691d5a61 not found: ID does not exist" containerID="b64c11fb315bcfd42cb83c284b4bcaa758951274e917ab740b702d4b691d5a61"
Sep 17 03:53:32.583077 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.583020    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b64c11fb315bcfd42cb83c284b4bcaa758951274e917ab740b702d4b691d5a61"} err="failed to get container status \"b64c11fb315bcfd42cb83c284b4bcaa758951274e917ab740b702d4b691d5a61\": rpc error: code = NotFound desc = could not find container \"b64c11fb315bcfd42cb83c284b4bcaa758951274e917ab740b702d4b691d5a61\": container with ID starting with b64c11fb315bcfd42cb83c284b4bcaa758951274e917ab740b702d4b691d5a61 not found: ID does not exist"
Sep 17 03:53:32.583077 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.583044    2576 scope.go:117] "RemoveContainer" containerID="1293da3aaa0b7e209dcbab3cb07f88dbb972e37e87a26af4717e325d59f51ed3"
Sep 17 03:53:32.583831 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.583810    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-22e045-6bb4d5bcf9-n2fl7"]
Sep 17 03:53:32.591297 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.591268    2576 scope.go:117] "RemoveContainer" containerID="1293da3aaa0b7e209dcbab3cb07f88dbb972e37e87a26af4717e325d59f51ed3"
Sep 17 03:53:32.591615 ip-10-0-137-141 kubenswrapper[2576]: E0917 03:53:32.591588    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1293da3aaa0b7e209dcbab3cb07f88dbb972e37e87a26af4717e325d59f51ed3\": container with ID starting with 1293da3aaa0b7e209dcbab3cb07f88dbb972e37e87a26af4717e325d59f51ed3 not found: ID does not exist" containerID="1293da3aaa0b7e209dcbab3cb07f88dbb972e37e87a26af4717e325d59f51ed3"
Sep 17 03:53:32.591691 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.591623    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1293da3aaa0b7e209dcbab3cb07f88dbb972e37e87a26af4717e325d59f51ed3"} err="failed to get container status \"1293da3aaa0b7e209dcbab3cb07f88dbb972e37e87a26af4717e325d59f51ed3\": rpc error: code = NotFound desc = could not find container \"1293da3aaa0b7e209dcbab3cb07f88dbb972e37e87a26af4717e325d59f51ed3\": container with ID starting with 1293da3aaa0b7e209dcbab3cb07f88dbb972e37e87a26af4717e325d59f51ed3 not found: ID does not exist"
Sep 17 03:53:32.595029 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.595006    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"]
Sep 17 03:53:32.598391 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.598371    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-22e045-64b6b5bd5d-7xzxv"]
Sep 17 03:53:32.609559 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.609538    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"]
Sep 17 03:53:32.613817 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:32.611873    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-22e045-567cd6d477-rcbkh"]
Sep 17 03:53:33.180790 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:33.180708    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:33.986276 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:33.986240    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:34.044396 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:34.044367    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:34.293739 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:34.293651    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f7764ac-5295-4921-aa0c-2e132638125c" path="/var/lib/kubelet/pods/3f7764ac-5295-4921-aa0c-2e132638125c/volumes"
Sep 17 03:53:34.294089 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:34.294005    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="941c982b-fa6f-4f6f-b088-589bf23e7162" path="/var/lib/kubelet/pods/941c982b-fa6f-4f6f-b088-589bf23e7162/volumes"
Sep 17 03:53:34.294339 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:34.294324    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc41ede3-20c0-40b8-a131-f8979f03a1e1" path="/var/lib/kubelet/pods/fc41ede3-20c0-40b8-a131-f8979f03a1e1/volumes"
Sep 17 03:53:35.226135 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:35.226099    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:53:38.290388 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.290356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:38.373705 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.373664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:38.597173 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.597132    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:38.926891 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.926809    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq"]
Sep 17 03:53:38.927125 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.927113    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f7764ac-5295-4921-aa0c-2e132638125c" containerName="payload-pre-processing"
Sep 17 03:53:38.927169 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.927127    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f7764ac-5295-4921-aa0c-2e132638125c" containerName="payload-pre-processing"
Sep 17 03:53:38.927169 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.927146    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d20faa63-e131-4865-b0a9-bbeabe709b35" containerName="revoke-keys"
Sep 17 03:53:38.927169 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.927152    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d20faa63-e131-4865-b0a9-bbeabe709b35" containerName="revoke-keys"
Sep 17 03:53:38.927169 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.927161    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="941c982b-fa6f-4f6f-b088-589bf23e7162" containerName="payload-processing"
Sep 17 03:53:38.927169 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.927167    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="941c982b-fa6f-4f6f-b088-589bf23e7162" containerName="payload-processing"
Sep 17 03:53:38.927322 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.927173    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc41ede3-20c0-40b8-a131-f8979f03a1e1" containerName="maas-api"
Sep 17 03:53:38.927322 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.927178    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc41ede3-20c0-40b8-a131-f8979f03a1e1" containerName="maas-api"
Sep 17 03:53:38.927322 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.927225    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="941c982b-fa6f-4f6f-b088-589bf23e7162" containerName="payload-processing"
Sep 17 03:53:38.927322 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.927231    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f7764ac-5295-4921-aa0c-2e132638125c" containerName="payload-pre-processing"
Sep 17 03:53:38.927322 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.927239    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc41ede3-20c0-40b8-a131-f8979f03a1e1" containerName="maas-api"
Sep 17 03:53:38.927322 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.927245    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d20faa63-e131-4865-b0a9-bbeabe709b35" containerName="revoke-keys"
Sep 17 03:53:38.930097 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.930079    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq"
Sep 17 03:53:38.932312 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.932291    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 03:53:38.932457 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.932295    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8vwnw\""
Sep 17 03:53:38.939486 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:38.939461    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq"]
Sep 17 03:53:39.050800 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:39.050751    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq\" (UID: \"338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq"
Sep 17 03:53:39.152178 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:39.152133    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq\" (UID: \"338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq"
Sep 17 03:53:39.152823 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:39.152802    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq\" (UID: \"338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq"
Sep 17 03:53:39.240767 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:39.240679    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq"
Sep 17 03:53:39.259410 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:39.259383    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:39.517516 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:39.517491    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq"]
Sep 17 03:53:39.519994 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:53:39.519962    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338d9d5f_1e65_4c6a_932f_0fa7bdc1f7f7.slice/crio-ab9133e6d91d361a3ce415a46b2b149435350e808c5ec0c8227e94b8679aa159 WatchSource:0}: Error finding container ab9133e6d91d361a3ce415a46b2b149435350e808c5ec0c8227e94b8679aa159: Status 404 returned error can't find the container with id ab9133e6d91d361a3ce415a46b2b149435350e808c5ec0c8227e94b8679aa159
Sep 17 03:53:39.601265 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:39.601229    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq" event={"ID":"338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7","Type":"ContainerStarted","Data":"ab9133e6d91d361a3ce415a46b2b149435350e808c5ec0c8227e94b8679aa159"}
Sep 17 03:53:40.079637 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:40.079598    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:40.607763 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:40.607733    2576 generic.go:356] "Generic (PLEG): container finished" podID="338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7" containerID="af6ba0de14adf77fa744a7d3901fd50b122a384fd25788ab49bb46431ae6d376" exitCode=0
Sep 17 03:53:40.608156 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:40.607817    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq" event={"ID":"338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7","Type":"ContainerDied","Data":"af6ba0de14adf77fa744a7d3901fd50b122a384fd25788ab49bb46431ae6d376"}
Sep 17 03:53:40.724870 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:40.724837    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:41.745372 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:41.745349    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq"
Sep 17 03:53:41.875060 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:41.874955    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7-maas-api-service-ca\") pod \"338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7\" (UID: \"338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7\") "
Sep 17 03:53:41.875381 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:41.875353    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7" (UID: "338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 03:53:41.975734 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:41.975696    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7-maas-api-service-ca\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:53:42.619566 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:42.619482    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq" event={"ID":"338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7","Type":"ContainerDied","Data":"ab9133e6d91d361a3ce415a46b2b149435350e808c5ec0c8227e94b8679aa159"}
Sep 17 03:53:42.619566 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:42.619502    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq"
Sep 17 03:53:42.619566 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:42.619533    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab9133e6d91d361a3ce415a46b2b149435350e808c5ec0c8227e94b8679aa159"
Sep 17 03:53:49.425764 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:49.425715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:49.487034 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:49.487005    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:49.585386 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:49.585352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:49.611255 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:49.611216    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq"]
Sep 17 03:53:49.613330 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:49.613296    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-f6d581-g9lfq"]
Sep 17 03:53:49.640181 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:49.640154    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:49.988160 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:49.988126    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:50.088597 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:50.088561    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:50.295136 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:50.295061    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7" path="/var/lib/kubelet/pods/338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7/volumes"
Sep 17 03:53:52.032079 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:52.032041    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:53:55.885180 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:55.885095    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:55.952985 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:55.952925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:53:56.157119 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:53:56.157028    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:54:01.529457 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:54:01.529409    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:54:32.382570 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:54:32.382540    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:54:32.386313 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:54:32.386288    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:55:19.248248 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:55:19.248210    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:55:20.134566 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:55:20.134535    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:55:21.131054 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:55:21.131016    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:55:22.275669 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:55:22.275640    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:55:23.859604 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:55:23.859571    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:55:47.372256 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:55:47.372226    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:55:47.472290 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:55:47.472257    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:56:01.533831 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:01.533792    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:56:07.234248 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:07.234204    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:56:13.631142 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:13.631106    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:56:21.939062 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:21.939026    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:56:39.099276 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:39.099241    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:56:40.639255 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:40.639217    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:56:49.225500 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:49.225460    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:56:52.018976 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:52.018943    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"]
Sep 17 03:56:52.019475 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:52.019166    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx" podUID="b16dffd3-95d8-4fa6-ac56-fb30275e4c24" containerName="manager" containerID="cri-o://1e5050bd92532cc447bc5ec43d834236cae44b5a30efe99b961c3470ec0c5f14" gracePeriod=10
Sep 17 03:56:53.616001 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:53.615967    2576 generic.go:356] "Generic (PLEG): container finished" podID="b16dffd3-95d8-4fa6-ac56-fb30275e4c24" containerID="1e5050bd92532cc447bc5ec43d834236cae44b5a30efe99b961c3470ec0c5f14" exitCode=0
Sep 17 03:56:53.616466 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:53.616046    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx" event={"ID":"b16dffd3-95d8-4fa6-ac56-fb30275e4c24","Type":"ContainerDied","Data":"1e5050bd92532cc447bc5ec43d834236cae44b5a30efe99b961c3470ec0c5f14"}
Sep 17 03:56:53.616466 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:53.616125    2576 scope.go:117] "RemoveContainer" containerID="59f827834ae9772fa54f3b9bad7d4ad30fb78d7760c2b082c84a3f9d658fe595"
Sep 17 03:56:54.163619 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:54.163594    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:56:54.282342 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:54.282313    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b16dffd3-95d8-4fa6-ac56-fb30275e4c24-extensions-socket-volume\") pod \"b16dffd3-95d8-4fa6-ac56-fb30275e4c24\" (UID: \"b16dffd3-95d8-4fa6-ac56-fb30275e4c24\") "
Sep 17 03:56:54.282577 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:54.282357    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktd5v\" (UniqueName: \"kubernetes.io/projected/b16dffd3-95d8-4fa6-ac56-fb30275e4c24-kube-api-access-ktd5v\") pod \"b16dffd3-95d8-4fa6-ac56-fb30275e4c24\" (UID: \"b16dffd3-95d8-4fa6-ac56-fb30275e4c24\") "
Sep 17 03:56:54.282723 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:54.282701    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b16dffd3-95d8-4fa6-ac56-fb30275e4c24-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b16dffd3-95d8-4fa6-ac56-fb30275e4c24" (UID: "b16dffd3-95d8-4fa6-ac56-fb30275e4c24"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 03:56:54.284381 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:54.284334    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b16dffd3-95d8-4fa6-ac56-fb30275e4c24-kube-api-access-ktd5v" (OuterVolumeSpecName: "kube-api-access-ktd5v") pod "b16dffd3-95d8-4fa6-ac56-fb30275e4c24" (UID: "b16dffd3-95d8-4fa6-ac56-fb30275e4c24"). InnerVolumeSpecName "kube-api-access-ktd5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 03:56:54.383562 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:54.383522    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b16dffd3-95d8-4fa6-ac56-fb30275e4c24-extensions-socket-volume\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:56:54.383562 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:54.383552    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktd5v\" (UniqueName: \"kubernetes.io/projected/b16dffd3-95d8-4fa6-ac56-fb30275e4c24-kube-api-access-ktd5v\") on node \"ip-10-0-137-141.ec2.internal\" DevicePath \"\""
Sep 17 03:56:54.623007 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:54.622977    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"
Sep 17 03:56:54.623509 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:54.623023    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx" event={"ID":"b16dffd3-95d8-4fa6-ac56-fb30275e4c24","Type":"ContainerDied","Data":"ebd7fd71451ee09e583eace7004eb18e7786187f3c185377bd1646e67e94cac8"}
Sep 17 03:56:54.623509 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:54.623086    2576 scope.go:117] "RemoveContainer" containerID="1e5050bd92532cc447bc5ec43d834236cae44b5a30efe99b961c3470ec0c5f14"
Sep 17 03:56:54.639553 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:54.639527    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"]
Sep 17 03:56:54.643626 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:54.643604    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ctzbx"]
Sep 17 03:56:56.293694 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:56:56.293650    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b16dffd3-95d8-4fa6-ac56-fb30275e4c24" path="/var/lib/kubelet/pods/b16dffd3-95d8-4fa6-ac56-fb30275e4c24/volumes"
Sep 17 03:57:52.693979 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.693945    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw"]
Sep 17 03:57:52.694409 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.694252    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b16dffd3-95d8-4fa6-ac56-fb30275e4c24" containerName="manager"
Sep 17 03:57:52.694409 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.694263    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b16dffd3-95d8-4fa6-ac56-fb30275e4c24" containerName="manager"
Sep 17 03:57:52.694409 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.694277    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b16dffd3-95d8-4fa6-ac56-fb30275e4c24" containerName="manager"
Sep 17 03:57:52.694409 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.694283    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b16dffd3-95d8-4fa6-ac56-fb30275e4c24" containerName="manager"
Sep 17 03:57:52.694409 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.694293    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7" containerName="revoke-keys"
Sep 17 03:57:52.694409 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.694299    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7" containerName="revoke-keys"
Sep 17 03:57:52.694409 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.694354    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b16dffd3-95d8-4fa6-ac56-fb30275e4c24" containerName="manager"
Sep 17 03:57:52.694409 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.694363    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b16dffd3-95d8-4fa6-ac56-fb30275e4c24" containerName="manager"
Sep 17 03:57:52.694409 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.694371    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="338d9d5f-1e65-4c6a-932f-0fa7bdc1f7f7" containerName="revoke-keys"
Sep 17 03:57:52.697323 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.697304    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw"
Sep 17 03:57:52.699400 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.699383    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2tz2l\""
Sep 17 03:57:52.711515 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.711489    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw"]
Sep 17 03:57:52.779672 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.779633    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zlrr\" (UniqueName: \"kubernetes.io/projected/aa5f14b6-1715-41ec-8d4b-20d0b1c1c906-kube-api-access-8zlrr\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mzrsw\" (UID: \"aa5f14b6-1715-41ec-8d4b-20d0b1c1c906\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw"
Sep 17 03:57:52.779888 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.779721    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa5f14b6-1715-41ec-8d4b-20d0b1c1c906-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mzrsw\" (UID: \"aa5f14b6-1715-41ec-8d4b-20d0b1c1c906\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw"
Sep 17 03:57:52.880906 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.880872    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa5f14b6-1715-41ec-8d4b-20d0b1c1c906-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mzrsw\" (UID: \"aa5f14b6-1715-41ec-8d4b-20d0b1c1c906\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw"
Sep 17 03:57:52.881085 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.880946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zlrr\" (UniqueName: \"kubernetes.io/projected/aa5f14b6-1715-41ec-8d4b-20d0b1c1c906-kube-api-access-8zlrr\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mzrsw\" (UID: \"aa5f14b6-1715-41ec-8d4b-20d0b1c1c906\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw"
Sep 17 03:57:52.881332 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.881310    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa5f14b6-1715-41ec-8d4b-20d0b1c1c906-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mzrsw\" (UID: \"aa5f14b6-1715-41ec-8d4b-20d0b1c1c906\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw"
Sep 17 03:57:52.888605 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:52.888577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zlrr\" (UniqueName: \"kubernetes.io/projected/aa5f14b6-1715-41ec-8d4b-20d0b1c1c906-kube-api-access-8zlrr\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mzrsw\" (UID: \"aa5f14b6-1715-41ec-8d4b-20d0b1c1c906\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw"
Sep 17 03:57:53.006858 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:53.006764    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw"
Sep 17 03:57:53.284416 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:53.284387    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw"]
Sep 17 03:57:53.285843 ip-10-0-137-141 kubenswrapper[2576]: W0917 03:57:53.285808    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa5f14b6_1715_41ec_8d4b_20d0b1c1c906.slice/crio-072c3f19d15511335814f39a9b1dc48cc8abff99eaf8ad5528ea705581a1bb2f WatchSource:0}: Error finding container 072c3f19d15511335814f39a9b1dc48cc8abff99eaf8ad5528ea705581a1bb2f: Status 404 returned error can't find the container with id 072c3f19d15511335814f39a9b1dc48cc8abff99eaf8ad5528ea705581a1bb2f
Sep 17 03:57:53.929977 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:53.929940    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw" event={"ID":"aa5f14b6-1715-41ec-8d4b-20d0b1c1c906","Type":"ContainerStarted","Data":"21fafc778bd65e5656633765c66f7367ea2d060d177d76ebe501d86dfb8806af"}
Sep 17 03:57:53.930447 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:53.930029    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw" event={"ID":"aa5f14b6-1715-41ec-8d4b-20d0b1c1c906","Type":"ContainerStarted","Data":"072c3f19d15511335814f39a9b1dc48cc8abff99eaf8ad5528ea705581a1bb2f"}
Sep 17 03:57:53.930447 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:53.930148    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw"
Sep 17 03:57:53.948868 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:57:53.948814    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw" podStartSLOduration=1.948793297 podStartE2EDuration="1.948793297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 03:57:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 03:57:53.947501243 +0000 UTC m=+2302.292227323" watchObservedRunningTime="2026-09-17 03:57:53.948793297 +0000 UTC m=+2302.293519378"
Sep 17 03:58:04.938271 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:58:04.938236    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzrsw"
Sep 17 03:58:09.849607 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:58:09.849564    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:58:09.899962 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:58:09.899936    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:58:15.634117 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:58:15.634080    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:58:22.829273 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:58:22.829234    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:58:23.436064 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:58:23.436020    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:58:31.627576 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:58:31.627496    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:58:42.713892 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:58:42.713864    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:58:45.224298 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:58:45.224252    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:58:48.988547 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:58:48.988517    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:58:52.530860 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:58:52.530828    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:59:19.478959 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:59:19.478900    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:59:20.265704 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:59:20.265664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:59:20.366090 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:59:20.366048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:59:21.080850 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:59:21.080818    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 03:59:21.829752 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:59:21.829716    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:59:30.633073 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:59:30.633035    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:59:32.416883 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:59:32.416845    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:59:32.422515 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:59:32.422494    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 03:59:32.667807 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:59:32.667718    2576 scope.go:117] "RemoveContainer" containerID="47d8421167d66b9bff4a0dad19d03306a5e1849213d0509205a214d62344a93f"
Sep 17 03:59:32.676280 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:59:32.676261    2576 scope.go:117] "RemoveContainer" containerID="5b869ae825ed3d7d27c486926a662da8a76f4038bbee43afbc76c09be11b8fac"
Sep 17 03:59:39.626968 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:59:39.626931    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:59:47.239158 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:59:47.239119    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 03:59:54.623960 ip-10-0-137-141 kubenswrapper[2576]: I0917 03:59:54.623875    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 04:00:00.125041 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:00.125001    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826960-8b22t"]
Sep 17 04:00:00.128519 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:00.128502    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826960-8b22t"
Sep 17 04:00:00.130624 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:00.130597    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8vwnw\""
Sep 17 04:00:00.135194 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:00.135161    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826960-8b22t"]
Sep 17 04:00:00.139244 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:00.139221    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826960-8b22t"
Sep 17 04:00:00.408864 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:00.408836    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826960-8b22t"]
Sep 17 04:00:00.410400 ip-10-0-137-141 kubenswrapper[2576]: W0917 04:00:00.410355    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31781ad5_4aaa_48ca_b52e_d6447aeddef2.slice/crio-5ea3a0f43217903446dc82c5c8be09bba47123232fb8b3707fad8099f059bea4 WatchSource:0}: Error finding container 5ea3a0f43217903446dc82c5c8be09bba47123232fb8b3707fad8099f059bea4: Status 404 returned error can't find the container with id 5ea3a0f43217903446dc82c5c8be09bba47123232fb8b3707fad8099f059bea4
Sep 17 04:00:00.587266 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:00.587233    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826960-8b22t" event={"ID":"31781ad5-4aaa-48ca-b52e-d6447aeddef2","Type":"ContainerStarted","Data":"5ea3a0f43217903446dc82c5c8be09bba47123232fb8b3707fad8099f059bea4"}
Sep 17 04:00:01.594872 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:01.594833    2576 generic.go:356] "Generic (PLEG): container finished" podID="31781ad5-4aaa-48ca-b52e-d6447aeddef2" containerID="0322825671eede7504dee427a753a7aa4703831fd6168b567d2c3fac4b4b5e8c" exitCode=0
Sep 17 04:00:01.595246 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:01.594878    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826960-8b22t" event={"ID":"31781ad5-4aaa-48ca-b52e-d6447aeddef2","Type":"ContainerDied","Data":"0322825671eede7504dee427a753a7aa4703831fd6168b567d2c3fac4b4b5e8c"}
Sep 17 04:00:02.725685 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:02.725658    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826960-8b22t"
Sep 17 04:00:03.030757 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:03.030664    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 04:00:03.607140 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:03.607093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826960-8b22t" event={"ID":"31781ad5-4aaa-48ca-b52e-d6447aeddef2","Type":"ContainerDied","Data":"5ea3a0f43217903446dc82c5c8be09bba47123232fb8b3707fad8099f059bea4"}
Sep 17 04:00:03.607140 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:03.607129    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826960-8b22t"
Sep 17 04:00:03.607140 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:03.607145    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ea3a0f43217903446dc82c5c8be09bba47123232fb8b3707fad8099f059bea4"
Sep 17 04:00:08.241774 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:08.241731    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-llwrp"]
Sep 17 04:00:16.911564 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:16.911531    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fqsqm_071b4aba-067f-47ab-b292-3234865c0eb3/server/0.log"
Sep 17 04:00:17.028982 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:17.028950    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-h64t5_d784a82c-aa35-4321-89ef-de3cee39c699/manager/0.log"
Sep 17 04:00:17.365880 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:17.365850    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6977f7b7bc-hs6g2_9fb7347b-fab7-4d3f-a416-2943865f87b3/copy-manifests/0.log"
Sep 17 04:00:17.372229 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:17.372208    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6977f7b7bc-hs6g2_9fb7347b-fab7-4d3f-a416-2943865f87b3/manager/0.log"
Sep 17 04:00:18.120772 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:18.120737    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29826960-8b22t_31781ad5-4aaa-48ca-b52e-d6447aeddef2/cleanup/0.log"
Sep 17 04:00:19.157472 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:19.157438    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-cbrdc_d41ae147-d2d2-4ac8-83c3-3d53ed056ca5/manager/0.log"
Sep 17 04:00:19.502412 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:19.502331    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-mzrsw_aa5f14b6-1715-41ec-8d4b-20d0b1c1c906/manager/0.log"
Sep 17 04:00:19.635914 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:19.635886    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-llwrp_d2e653b6-974a-4bdd-b84e-09f1832ce26f/limitador/0.log"
Sep 17 04:00:20.230181 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:20.230149    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-qwlft_70731ed8-7be6-45c7-b277-24e63a8ab980/discovery/0.log"
Sep 17 04:00:20.334332 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:20.334298    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bc6dc7888-57x9k_457fad5e-53c6-439b-9000-859da48c58db/kube-auth-proxy/0.log"
Sep 17 04:00:21.358189 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:21.358153    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-q9b6v_8175a1c8-b4a3-4a15-b5af-540e4cd1b645/main/0.log"
Sep 17 04:00:21.814728 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:21.814702    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-kq8z9_3fcc975f-291c-4f45-ba48-f9e54c161fdf/main/0.log"
Sep 17 04:00:26.856409 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:26.856373    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 04:00:29.755166 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:29.755120    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log" limit=1024
Sep 17 04:00:29.781266 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:29.781234    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9cp4t_2ed49a17-07c3-4f80-8a00-7b5f21fbb61c/global-pull-secret-syncer/0.log"
Sep 17 04:00:29.857227 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:29.857198    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-65r5r_cf8486c3-9f66-4dd8-b244-634629e430fe/konnectivity-agent/0.log"
Sep 17 04:00:29.955482 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:29.955449    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-141.ec2.internal_1c7703ee05308cc145211e33cf98cc0b/haproxy/0.log"
Sep 17 04:00:32.701227 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:32.701189    2576 scope.go:117] "RemoveContainer" containerID="af6ba0de14adf77fa744a7d3901fd50b122a384fd25788ab49bb46431ae6d376"
Sep 17 04:00:34.786214 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:34.786184    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-cbrdc_d41ae147-d2d2-4ac8-83c3-3d53ed056ca5/manager/0.log"
Sep 17 04:00:34.866884 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:34.866817    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-mzrsw_aa5f14b6-1715-41ec-8d4b-20d0b1c1c906/manager/0.log"
Sep 17 04:00:34.895939 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:34.895907    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-llwrp_d2e653b6-974a-4bdd-b84e-09f1832ce26f/limitador/0.log"
Sep 17 04:00:36.817980 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:36.817949    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mh687_672c0fb5-6604-47b5-b727-dd595e599157/node-exporter/0.log"
Sep 17 04:00:36.837895 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:36.837855    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mh687_672c0fb5-6604-47b5-b727-dd595e599157/kube-rbac-proxy/0.log"
Sep 17 04:00:36.863724 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:36.863699    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mh687_672c0fb5-6604-47b5-b727-dd595e599157/init-textfile/0.log"
Sep 17 04:00:39.993365 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:39.993330    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f94cc7445-h87bm_f4f6ef90-b889-467e-afa2-fd95f0f0087d/console/0.log"
Sep 17 04:00:40.153690 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.153656    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"]
Sep 17 04:00:40.153986 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.153973    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31781ad5-4aaa-48ca-b52e-d6447aeddef2" containerName="cleanup"
Sep 17 04:00:40.154032 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.153987    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="31781ad5-4aaa-48ca-b52e-d6447aeddef2" containerName="cleanup"
Sep 17 04:00:40.154068 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.154053    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="31781ad5-4aaa-48ca-b52e-d6447aeddef2" containerName="cleanup"
Sep 17 04:00:40.156863 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.156847    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.159032 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.159013    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nv4hh\"/\"kube-root-ca.crt\""
Sep 17 04:00:40.159614 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.159593    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nv4hh\"/\"default-dockercfg-8vt88\""
Sep 17 04:00:40.159614 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.159604    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nv4hh\"/\"openshift-service-ca.crt\""
Sep 17 04:00:40.173651 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.173619    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"]
Sep 17 04:00:40.214867 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.214825    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/242663fd-3d54-4eca-a2d7-63d7c9813275-lib-modules\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.215045 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.214873    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/242663fd-3d54-4eca-a2d7-63d7c9813275-sys\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.215045 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.214959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/242663fd-3d54-4eca-a2d7-63d7c9813275-podres\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.215045 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.215008    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8zcg\" (UniqueName: \"kubernetes.io/projected/242663fd-3d54-4eca-a2d7-63d7c9813275-kube-api-access-c8zcg\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.215045 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.215027    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/242663fd-3d54-4eca-a2d7-63d7c9813275-proc\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.315755 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.315725    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/242663fd-3d54-4eca-a2d7-63d7c9813275-lib-modules\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.315946 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.315771    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/242663fd-3d54-4eca-a2d7-63d7c9813275-sys\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.315946 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.315864    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/242663fd-3d54-4eca-a2d7-63d7c9813275-sys\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.315946 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.315917    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/242663fd-3d54-4eca-a2d7-63d7c9813275-lib-modules\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.315946 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.315924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/242663fd-3d54-4eca-a2d7-63d7c9813275-podres\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.316095 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.316010    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8zcg\" (UniqueName: \"kubernetes.io/projected/242663fd-3d54-4eca-a2d7-63d7c9813275-kube-api-access-c8zcg\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.316095 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.316046    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/242663fd-3d54-4eca-a2d7-63d7c9813275-proc\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.316157 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.316109    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/242663fd-3d54-4eca-a2d7-63d7c9813275-proc\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.316157 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.316112    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/242663fd-3d54-4eca-a2d7-63d7c9813275-podres\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.323741 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.323709    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8zcg\" (UniqueName: \"kubernetes.io/projected/242663fd-3d54-4eca-a2d7-63d7c9813275-kube-api-access-c8zcg\") pod \"perf-node-gather-daemonset-gxfvj\" (UID: \"242663fd-3d54-4eca-a2d7-63d7c9813275\") " pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.470028 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.469994    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.742114 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.742081    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"]
Sep 17 04:00:40.743008 ip-10-0-137-141 kubenswrapper[2576]: W0917 04:00:40.742975    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod242663fd_3d54_4eca_a2d7_63d7c9813275.slice/crio-608372beaccea9277d368fef79e98e5ec25deaa5fe6ebed84615f4b49fb330a2 WatchSource:0}: Error finding container 608372beaccea9277d368fef79e98e5ec25deaa5fe6ebed84615f4b49fb330a2: Status 404 returned error can't find the container with id 608372beaccea9277d368fef79e98e5ec25deaa5fe6ebed84615f4b49fb330a2
Sep 17 04:00:40.805810 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.805778    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj" event={"ID":"242663fd-3d54-4eca-a2d7-63d7c9813275","Type":"ContainerStarted","Data":"d002070988c6dc10647e78c91450bc8288b8b9c0658c649f06502a57b80a3d44"}
Sep 17 04:00:40.806015 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.805832    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj" event={"ID":"242663fd-3d54-4eca-a2d7-63d7c9813275","Type":"ContainerStarted","Data":"608372beaccea9277d368fef79e98e5ec25deaa5fe6ebed84615f4b49fb330a2"}
Sep 17 04:00:40.806015 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.805867    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:40.824042 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:40.823988    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj" podStartSLOduration=0.823972367 podStartE2EDuration="823.972367ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 04:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 04:00:40.822977499 +0000 UTC m=+2469.167703579" watchObservedRunningTime="2026-09-17 04:00:40.823972367 +0000 UTC m=+2469.168698446"
Sep 17 04:00:41.857186 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:41.857159    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7hqg9_b20e378f-f35f-420a-9b02-fd76bf5a2bd1/dns/0.log"
Sep 17 04:00:41.873891 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:41.873842    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7hqg9_b20e378f-f35f-420a-9b02-fd76bf5a2bd1/kube-rbac-proxy/0.log"
Sep 17 04:00:41.953301 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:41.953276    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f9mkc_05b2116c-7c2b-4424-8038-a5580cac87b2/dns-node-resolver/0.log"
Sep 17 04:00:42.543843 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:42.543815    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sp5r2_7b8eeb04-e37e-43eb-8f94-b0eeeb6fb158/node-ca/0.log"
Sep 17 04:00:43.836319 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:43.836286    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-qwlft_70731ed8-7be6-45c7-b277-24e63a8ab980/discovery/0.log"
Sep 17 04:00:43.853582 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:43.853555    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bc6dc7888-57x9k_457fad5e-53c6-439b-9000-859da48c58db/kube-auth-proxy/0.log"
Sep 17 04:00:44.836942 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:44.836914    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-44gmz_fcc07e33-2671-4c55-9211-9714c8799a6e/serve-healthcheck-canary/0.log"
Sep 17 04:00:45.446029 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:45.445998    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nxtgf_40fe1061-8f7b-4385-9c4f-75311f202ac7/kube-rbac-proxy/0.log"
Sep 17 04:00:45.462801 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:45.462767    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nxtgf_40fe1061-8f7b-4385-9c4f-75311f202ac7/exporter/0.log"
Sep 17 04:00:45.479388 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:45.479360    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nxtgf_40fe1061-8f7b-4385-9c4f-75311f202ac7/extractor/0.log"
Sep 17 04:00:46.821596 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:46.821565    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nv4hh/perf-node-gather-daemonset-gxfvj"
Sep 17 04:00:47.906346 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:47.906316    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fqsqm_071b4aba-067f-47ab-b292-3234865c0eb3/server/0.log"
Sep 17 04:00:47.994811 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:47.994780    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-h64t5_d784a82c-aa35-4321-89ef-de3cee39c699/manager/0.log"
Sep 17 04:00:48.159011 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:48.158927    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6977f7b7bc-hs6g2_9fb7347b-fab7-4d3f-a416-2943865f87b3/manager/0.log"
Sep 17 04:00:48.179476 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:48.179446    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6977f7b7bc-hs6g2_9fb7347b-fab7-4d3f-a416-2943865f87b3/copy-manifests/0.log"
Sep 17 04:00:49.411977 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:49.411946    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-s6t6d_0ae8e927-56ef-48da-951e-3f7e18968c68/openshift-lws-operator/0.log"
Sep 17 04:00:50.520549 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:50.520513    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-qwlft_70731ed8-7be6-45c7-b277-24e63a8ab980/discovery/0.log"
Sep 17 04:00:50.531276 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:50.531244    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bc6dc7888-57x9k_457fad5e-53c6-439b-9000-859da48c58db/kube-auth-proxy/0.log"
Sep 17 04:00:52.671404 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:52.671318    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-qwlft_70731ed8-7be6-45c7-b277-24e63a8ab980/discovery/0.log"
Sep 17 04:00:52.692387 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:52.692359    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bc6dc7888-57x9k_457fad5e-53c6-439b-9000-859da48c58db/kube-auth-proxy/0.log"
Sep 17 04:00:54.860482 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:54.860449    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-qv9xm_bbc0d1a4-1a90-4ee8-b668-9e7802ed999c/migrator/0.log"
Sep 17 04:00:54.878608 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:54.878575    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-qv9xm_bbc0d1a4-1a90-4ee8-b668-9e7802ed999c/graceful-termination/0.log"
Sep 17 04:00:56.529290 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:56.529257    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vwpxz_19037775-9be2-4a9a-aa31-9f1d9fe8aa74/kube-multus-additional-cni-plugins/0.log"
Sep 17 04:00:56.546566 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:56.546540    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vwpxz_19037775-9be2-4a9a-aa31-9f1d9fe8aa74/egress-router-binary-copy/0.log"
Sep 17 04:00:56.563673 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:56.563646    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vwpxz_19037775-9be2-4a9a-aa31-9f1d9fe8aa74/cni-plugins/0.log"
Sep 17 04:00:56.579682 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:56.579654    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vwpxz_19037775-9be2-4a9a-aa31-9f1d9fe8aa74/bond-cni-plugin/0.log"
Sep 17 04:00:56.596342 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:56.596321    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vwpxz_19037775-9be2-4a9a-aa31-9f1d9fe8aa74/routeoverride-cni/0.log"
Sep 17 04:00:56.613344 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:56.613323    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vwpxz_19037775-9be2-4a9a-aa31-9f1d9fe8aa74/whereabouts-cni-bincopy/0.log"
Sep 17 04:00:56.629630 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:56.629606    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vwpxz_19037775-9be2-4a9a-aa31-9f1d9fe8aa74/whereabouts-cni/0.log"
Sep 17 04:00:56.744007 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:56.743977    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zxpp2_c7063c41-ff67-4ce2-9d63-8cab1d847ba0/kube-multus/0.log"
Sep 17 04:00:56.797458 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:56.797356    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b24v9_b21d9f86-427c-4e22-89dd-f30258785552/network-metrics-daemon/0.log"
Sep 17 04:00:56.811639 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:56.811609    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b24v9_b21d9f86-427c-4e22-89dd-f30258785552/kube-rbac-proxy/0.log"
Sep 17 04:00:57.808239 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:57.808214    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-controller/0.log"
Sep 17 04:00:57.823524 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:57.823496    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/1.log"
Sep 17 04:00:57.823680 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:57.823540    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovn-acl-logging/0.log"
Sep 17 04:00:57.842777 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:57.842750    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/kube-rbac-proxy-node/0.log"
Sep 17 04:00:57.862892 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:57.862863    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/kube-rbac-proxy-ovn-metrics/0.log"
Sep 17 04:00:57.881745 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:57.881724    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/northd/0.log"
Sep 17 04:00:57.899824 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:57.899800    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/nbdb/0.log"
Sep 17 04:00:57.918150 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:57.918126    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/sbdb/0.log"
Sep 17 04:00:57.968368 ip-10-0-137-141 kubenswrapper[2576]: I0917 04:00:57.968336    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w969r_ba9064f2-792b-4c2b-b804-5bd64bd69fc5/ovnkube-controller/0.log"