Sep 04 23:36:51.639218 ip-10-0-139-127 systemd[1]: Starting Kubernetes Kubelet...
Sep 04 23:36:51.997410 ip-10-0-139-127 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 23:36:51.997410 ip-10-0-139-127 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 04 23:36:51.997410 ip-10-0-139-127 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 23:36:51.997410 ip-10-0-139-127 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 04 23:36:51.997964 ip-10-0-139-127 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 23:36:51.998267 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:51.998201    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 04 23:36:52.002570 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002555    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 23:36:52.002570 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002570    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002573    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002576    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002579    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002581    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002584    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002591    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002594    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002597    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002599    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002602    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002604    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002607    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002609    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002612    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002614    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002618    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002620    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002623    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002626    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 23:36:52.002631 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002628    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002631    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002633    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002636    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002639    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002642    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002644    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002647    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002649    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002652    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002654    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002657    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002659    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002662    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002664    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002667    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002669    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002673    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002675    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002679    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 23:36:52.003117 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002682    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002685    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002688    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002690    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002692    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002695    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002697    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002701    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002703    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002706    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002708    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002711    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002713    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002717    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002721    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002725    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002729    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002732    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002735    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002737    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 23:36:52.003628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002740    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002743    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002747    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002750    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002753    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002756    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002759    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002762    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002765    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002768    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002771    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002774    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002776    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002780    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002782    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002785    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002787    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002790    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002793    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 23:36:52.004132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002796    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002798    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002801    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002803    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002806    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002808    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002811    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002814    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002816    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002819    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002822    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002825    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002827    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002830    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002832    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002835    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002837    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002840    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.002842    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003565    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 23:36:52.004600 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003572    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003574    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003577    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003580    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003584    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003586    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003589    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003592    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003594    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003597    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003599    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003602    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003605    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003607    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003610    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003612    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003615    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003617    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003620    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003623    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 23:36:52.005101 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003626    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003628    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003631    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003634    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003638    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003642    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003645    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003647    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003650    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003653    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003655    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003658    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003660    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003663    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003666    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003668    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003670    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003673    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003678    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 23:36:52.005585 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003681    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003684    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003688    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003691    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003693    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003698    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003700    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003704    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003706    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003709    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003711    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003714    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003717    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003719    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003722    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003725    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003727    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003730    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003732    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 23:36:52.006060 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003736    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003738    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003741    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003743    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003746    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003748    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003751    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003753    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003755    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003758    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003760    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003763    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003766    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003769    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003771    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003774    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003776    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003779    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003781    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003784    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 23:36:52.006535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003786    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003789    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003791    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003794    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003796    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003798    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003801    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003803    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003806    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003808    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003810    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003813    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003815    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003818    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003820    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003822    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003825    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003827    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003830    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.003832    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 23:36:52.007063 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004192    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004197    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004199    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004202    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004206    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004208    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004211    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004213    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004216    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004219    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004222    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004225    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004227    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004230    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004232    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004235    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004237    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004240    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004242    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004245    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 23:36:52.007535 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004247    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004250    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004252    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004255    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004258    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004260    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004263    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004265    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004268    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004270    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004273    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004275    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004278    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004280    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004283    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004286    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004289    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004292    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004294    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004297    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 23:36:52.008029 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004299    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004302    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004304    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004307    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004312    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004315    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004319    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004323    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004328    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004332    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004335    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004340    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004344    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004347    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004349    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004352    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004354    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004357    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004359    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004362    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 23:36:52.008537 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004364    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004366    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004371    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004375    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004378    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004380    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004384    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004390    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004393    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004398    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004403    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004407    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004425    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004428    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004431    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004433    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004436    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004438    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004441    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 23:36:52.009020 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004443    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004445    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004449    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004452    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004456    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004459    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004461    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004464    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004467    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004470    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004472    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004475    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004478    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004481    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004486    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004490    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004495    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004500    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004503    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.004506    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 23:36:52.009473 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004578    2564 flags.go:64] FLAG: --address="0.0.0.0"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004612    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004619    2564 flags.go:64] FLAG: --anonymous-auth="true"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004624    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004628    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004632    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004636    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004640    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004643    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004647    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004652    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004658    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004663    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004667    2564 flags.go:64] FLAG: --cgroup-root=""
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004669    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004672    2564 flags.go:64] FLAG: --client-ca-file=""
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004675    2564 flags.go:64] FLAG: --cloud-provider="external"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004678    2564 flags.go:64] FLAG: --cluster-dns="[]"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004682    2564 flags.go:64] FLAG: --cluster-domain=""
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004684    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004687    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004690    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004694    2564 flags.go:64] FLAG: --container-log-max-files="5"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004697    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 04 23:36:52.009998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004701    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004704    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004707    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004710    2564 flags.go:64] FLAG: --contention-profiling="false"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004713    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004716    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004719    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004722    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004728    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004733    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004738    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004743    2564 flags.go:64] FLAG: --enable-load-reader="false"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004746    2564 flags.go:64] FLAG: --enable-server="true"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004750    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004754    2564 flags.go:64] FLAG: --event-burst="100"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004757    2564 flags.go:64] FLAG: --event-qps="50"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004760    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004763    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004766    2564 flags.go:64] FLAG: --eviction-hard=""
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004769    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004772    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004775    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004778    2564 flags.go:64] FLAG: --eviction-soft=""
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004781    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004783    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 04 23:36:52.010561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004786    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004789    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004791    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004794    2564 flags.go:64] FLAG: --fail-swap-on="true"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004797    2564 flags.go:64] FLAG: --feature-gates=""
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004801    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004803    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004807    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004812    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004817    2564 flags.go:64] FLAG: --healthz-port="10248"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004822    2564 flags.go:64] FLAG: --help="false"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004825    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004828    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004831    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004834    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004837    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004841    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004844    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004847    2564 flags.go:64] FLAG: --image-service-endpoint=""
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004849    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004853    2564 flags.go:64] FLAG: --kube-api-burst="100"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004856    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004859    2564 flags.go:64] FLAG: --kube-api-qps="50"
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004862    2564 flags.go:64] FLAG: --kube-reserved=""
Sep 04 23:36:52.011219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004865    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004868    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004871    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004874    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004876    2564 flags.go:64] FLAG: --lock-file=""
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004879    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004882    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004885    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004908    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004914    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004918    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004920    2564 flags.go:64] FLAG: --logging-format="text"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004923    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004926    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004929    2564 flags.go:64] FLAG: --manifest-url=""
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004934    2564 flags.go:64] FLAG: --manifest-url-header=""
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004946    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004949    2564 flags.go:64] FLAG: --max-open-files="1000000"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004953    2564 flags.go:64] FLAG: --max-pods="110"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004956    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004959    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004962    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004965    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004968    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004972    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 04 23:36:52.011792 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004974    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004984    2564 flags.go:64] FLAG: --node-status-max-images="50"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004989    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004994    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.004998    2564 flags.go:64] FLAG: --pod-cidr=""
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005002    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005007    2564 flags.go:64] FLAG: --pod-manifest-path=""
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005010    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005013    2564 flags.go:64] FLAG: --pods-per-core="0"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005016    2564 flags.go:64] FLAG: --port="10250"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005019    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005022    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07a588fa2591919a9"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005025    2564 flags.go:64] FLAG: --qos-reserved=""
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005028    2564 flags.go:64] FLAG: --read-only-port="10255"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005031    2564 flags.go:64] FLAG: --register-node="true"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005033    2564 flags.go:64] FLAG: --register-with-taints=""
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005037    2564 flags.go:64] FLAG: --registry-burst="10"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005040    2564 flags.go:64] FLAG: --registry-qps="5"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005042    2564 flags.go:64] FLAG: --reserved-cpus=""
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005045    2564 flags.go:64] FLAG: --reserved-memory=""
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005048    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005051    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005056    2564 flags.go:64] FLAG: --rotate-certificates="false"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005060    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005064    2564 flags.go:64] FLAG: --runonce="false"
Sep 04 23:36:52.012479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005069    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005074    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005078    2564 flags.go:64] FLAG: --seccomp-default="false"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005081    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005084    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005087    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005090    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005093    2564 flags.go:64] FLAG: --storage-driver-password="root"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005095    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005098    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005101    2564 flags.go:64] FLAG: --storage-driver-user="root"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005104    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005107    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005111    2564 flags.go:64] FLAG: --system-cgroups=""
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005113    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005118    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005121    2564 flags.go:64] FLAG: --tls-cert-file=""
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005124    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005128    2564 flags.go:64] FLAG: --tls-min-version=""
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005130    2564 flags.go:64] FLAG: --tls-private-key-file=""
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005133    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005136    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005139    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005144    2564 flags.go:64] FLAG: --v="2"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005150    2564 flags.go:64] FLAG: --version="false"
Sep 04 23:36:52.013105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005156    2564 flags.go:64] FLAG: --vmodule=""
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005159    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005163    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005295    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005303    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005308    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005312    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005316    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005319    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005322    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005325    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005327    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005330    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005334    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005337    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005340    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005342    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005345    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005347    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005350    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 23:36:52.013723 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005352    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005355    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005358    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005360    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005363    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005365    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005368    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005370    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005373    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005375    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005378    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005381    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005385    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005389    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005396    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005399    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005404    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005407    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005409    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005412    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 23:36:52.014231 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005415    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005417    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005420    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005423    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005426    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005429    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005432    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005435    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005438    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005440    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005443    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005445    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005448    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005452    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005456    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005460    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005465    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005469    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005474    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 23:36:52.014703 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005478    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005481    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005484    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005486    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005489    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005491    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005494    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005496    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005498    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005502    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005504    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005507    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005509    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005512    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005514    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005516    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005519    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005521    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005524    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005527    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 23:36:52.015177 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005529    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005532    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005534    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005537    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005541    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005547    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005554    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005558    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005562    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005564    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005567    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005569    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005572    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005574    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005577    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005579    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005582    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005584    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005587    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 23:36:52.015657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005590    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005592    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005596    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.005598    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.005603    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.012630    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.012644    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.012662    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.012670    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012727    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012732    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012735    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012738    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012741    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012744    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 23:36:52.016132 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012747    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012750    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012754    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012758    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012761    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012764    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012767    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012769    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012772    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012774    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012777    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012779    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012782    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012784    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012787    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012789    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012792    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012794    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012796    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 23:36:52.016524 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012799    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012802    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012804    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012807    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012810    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012813    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012816    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012819    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012821    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012824    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012826    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012829    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012831    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012834    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012836    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012839    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012841    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012844    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012846    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012849    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 23:36:52.017045 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012851    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012853    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012856    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012858    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012861    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012863    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012865    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012868    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012871    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012873    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012876    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012878    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012881    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012883    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012886    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012889    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012891    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012915    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012919    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012921    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 23:36:52.017549 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012924    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012926    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012929    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012932    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012934    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012946    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012950    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012953    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012956    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012959    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012961    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012963    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012966    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012968    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012971    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012974    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012976    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012979    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012981    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 23:36:52.018116 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012983    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012986    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012988    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012991    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012993    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012995    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.012998    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013001    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013003    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013006    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013009    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013012    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013015    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013017    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013021    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 23:36:52.018566 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.013027    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013128    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013133    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013136    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013139    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013142    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013145    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013147    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013150    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013152    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013155    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013159    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013163    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013166    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013169    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013172    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013175    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013177    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013180    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013182    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 23:36:52.018941 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013185    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013187    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013189    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013192    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013195    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013198    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013200    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013202    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013205    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013208    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013210    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013213    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013215    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013217    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013220    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013222    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013224    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013227    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013229    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013232    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 23:36:52.019420 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013234    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013236    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013239    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013241    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013244    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013246    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013248    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013251    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013253    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013255    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013258    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013261    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013263    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013266    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013268    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013270    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013273    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013275    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013278    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013280    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 23:36:52.019886 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013283    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013285    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013288    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013290    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013293    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013295    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013297    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013300    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013302    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013305    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013307    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013309    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013313    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013317    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013320    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013322    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013325    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013328    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013330    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013333    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 23:36:52.020409 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013336    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013338    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013341    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013343    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013345    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013348    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013350    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013353    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013357    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013360    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013362    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013364    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013367    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013369    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013372    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013374    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013377    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013379    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013382    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 23:36:52.020890 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:52.013384    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 23:36:52.021370 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.013389    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 23:36:52.021370 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.013477    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 04 23:36:52.021370 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.016155    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 04 23:36:52.021370 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.016952    2564 server.go:1019] "Starting client certificate rotation"
Sep 04 23:36:52.021370 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.017051    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 04 23:36:52.021370 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.017084    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 04 23:36:52.021370 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.020147    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 04 23:36:52.025456 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.025436    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 04 23:36:52.034242 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.034222    2564 log.go:25] "Validated CRI v1 runtime API"
Sep 04 23:36:52.037098 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.037082    2564 log.go:25] "Validated CRI v1 image API"
Sep 04 23:36:52.037868 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.037855    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 04 23:36:52.040174 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.040158    2564 fs.go:135] Filesystem UUIDs: map[7409df04-0fb1-487e-9c9b-20a076ed306b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 bbacfeba-846d-4739-b30b-23740988680c:/dev/nvme0n1p4]
Sep 04 23:36:52.040236 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.040174    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 04 23:36:52.045092 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.045066    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 04 23:36:52.045199 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.045100    2564 manager.go:216] Machine: {Timestamp:2026-09-04 23:36:52.043961838 +0000 UTC m=+0.291962356 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102850 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec281d58a008625e803b394a98ea05b6 SystemUUID:ec281d58-a008-625e-803b-394a98ea05b6 BootID:7a80cbbe-2c0e-496a-8258-b0945b39353d Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c5:40:16:d4:89 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c5:40:16:d4:89 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:22:cf:b5:01:e6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 04 23:36:52.045199 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.045196    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 04 23:36:52.045296 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.045266    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 04 23:36:52.048130 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.048103    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 04 23:36:52.048261 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.048133    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-127.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 04 23:36:52.049217 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.049206    2564 topology_manager.go:138] "Creating topology manager with none policy"
Sep 04 23:36:52.049254 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.049220    2564 container_manager_linux.go:309] "Creating device plugin manager"
Sep 04 23:36:52.049254 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.049232    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 04 23:36:52.049921 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.049911    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 04 23:36:52.050716 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.050707    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 04 23:36:52.053387 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.053376    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 04 23:36:52.053495 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.053486    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 04 23:36:52.053539 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.053514    2564 kubelet.go:486] "Attempting to sync node with API server"
Sep 04 23:36:52.053539 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.053524    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 04 23:36:52.053624 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.053541    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 04 23:36:52.053624 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.053551    2564 kubelet.go:398] "Adding apiserver pod source"
Sep 04 23:36:52.053624 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.053560    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 04 23:36:52.053861 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.053850    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 04 23:36:52.053938 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.053875    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 04 23:36:52.055761 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.055743    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 04 23:36:52.057801 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.057789    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 04 23:36:52.057850 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.057818    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 04 23:36:52.059401 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.059383    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 04 23:36:52.059454 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.059409    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 04 23:36:52.059454 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.059415    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 04 23:36:52.059454 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.059420    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 04 23:36:52.059454 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.059428    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 04 23:36:52.059454 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.059434    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 04 23:36:52.059454 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.059447    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 04 23:36:52.059454 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.059452    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 04 23:36:52.059634 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.059459    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 04 23:36:52.059634 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.059464    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 04 23:36:52.059634 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.059472    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 04 23:36:52.059634 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.059478    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 04 23:36:52.060674 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.060660    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 04 23:36:52.060710 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.060676    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 04 23:36:52.063407 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.063354    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 04 23:36:52.063509 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.063428    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-127.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 04 23:36:52.065127 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.064954    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-127.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 04 23:36:52.065211 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.065082    2564 server.go:1268] "Started kubelet"
Sep 04 23:36:52.065211 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.065148    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 04 23:36:52.065306 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.065194    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 04 23:36:52.065368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.065351    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 04 23:36:52.065976 ip-10-0-139-127 systemd[1]: Started Kubernetes Kubelet.
Sep 04 23:36:52.066376 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.066354    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 04 23:36:52.068510 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.068494    2564 server.go:310] "Adding debug handlers to kubelet server"
Sep 04 23:36:52.068983 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.068931    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8rljz"
Sep 04 23:36:52.069856 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.069838    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 04 23:36:52.072221 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.072207    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 04 23:36:52.072703 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.072686    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 04 23:36:52.072795 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.072751    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:52.073360 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.072512    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-127.ec2.internal.18d24266f9d39c2b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-127.ec2.internal,UID:ip-10-0-139-127.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-127.ec2.internal,},FirstTimestamp:2026-09-04 23:36:52.065000491 +0000 UTC m=+0.313000998,LastTimestamp:2026-09-04 23:36:52.065000491 +0000 UTC m=+0.313000998,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-127.ec2.internal,}"
Sep 04 23:36:52.073443 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.073359    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 04 23:36:52.073517 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.073435    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 04 23:36:52.073551 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.073529    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 04 23:36:52.073614 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.073603    2564 reconstruct.go:97] "Volume reconstruction finished"
Sep 04 23:36:52.073614 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.073613    2564 reconciler.go:29] "Reconciler: start to sync state"
Sep 04 23:36:52.074788 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.074763    2564 factory.go:55] Registering systemd factory
Sep 04 23:36:52.074788 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.074777    2564 factory.go:223] Registration of the systemd container factory successfully
Sep 04 23:36:52.075151 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.075137    2564 factory.go:153] Registering CRI-O factory
Sep 04 23:36:52.075203 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.075155    2564 factory.go:223] Registration of the crio container factory successfully
Sep 04 23:36:52.075203 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.075198    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 04 23:36:52.075274 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.075211    2564 factory.go:103] Registering Raw factory
Sep 04 23:36:52.075274 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.075228    2564 manager.go:1196] Started watching for new ooms in manager
Sep 04 23:36:52.075738 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.075723    2564 manager.go:349] Starting recovery of all containers
Sep 04 23:36:52.078461 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.078436    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8rljz"
Sep 04 23:36:52.082598 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.082565    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-127.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 04 23:36:52.082709 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.082657    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 04 23:36:52.086772 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.086757    2564 manager.go:354] Recovery completed
Sep 04 23:36:52.088308 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.088284    2564 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 04 23:36:52.091337 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.091321    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 23:36:52.093565 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.093550    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 23:36:52.093619 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.093579    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 23:36:52.093619 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.093592    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Sep 04 23:36:52.094090 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.094074    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 04 23:36:52.094090 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.094089    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 04 23:36:52.094186 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.094103    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 04 23:36:52.094186 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.094120    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 23:36:52.095012 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.094993    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 23:36:52.095233 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.095214    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 23:36:52.095313 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.095256    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Sep 04 23:36:52.095384 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.095217    2564 policy_none.go:49] "None policy: Start"
Sep 04 23:36:52.095493 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.095483    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 04 23:36:52.095612 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.095601    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 04 23:36:52.097198 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.097182    2564 policy_none.go:47] "Start"
Sep 04 23:36:52.134952 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.132551    2564 manager.go:337] "Starting Device Plugin manager"
Sep 04 23:36:52.134952 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.132584    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 04 23:36:52.134952 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.132597    2564 server.go:87] "Starting device plugin registration server"
Sep 04 23:36:52.134952 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.132890    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 04 23:36:52.134952 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.132922    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 04 23:36:52.134952 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.133026    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 04 23:36:52.134952 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.133102    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 04 23:36:52.134952 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.133111    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 04 23:36:52.134952 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.133564    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 04 23:36:52.134952 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.133607    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:52.166595 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.166574    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 04 23:36:52.167856 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.167841    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 04 23:36:52.167952 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.167892    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 04 23:36:52.167952 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.167933    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 04 23:36:52.168039 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.167974    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 04 23:36:52.169766 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.169748    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 23:36:52.233005 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.232983    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 23:36:52.233774 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.233758    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 23:36:52.233841 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.233786    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 23:36:52.233841 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.233802    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Sep 04 23:36:52.233841 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.233823    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.239196 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.239184    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.239237 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.239201    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-127.ec2.internal\": node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:52.252833 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.252782    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:52.268858 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.268839    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"]
Sep 04 23:36:52.268946 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.268930    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 23:36:52.271275 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.271261    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 23:36:52.271331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.271290    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 23:36:52.271331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.271300    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Sep 04 23:36:52.272339 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.272327    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 23:36:52.272466 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.272453    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.272499 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.272485    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 23:36:52.272992 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.272975    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 23:36:52.273134 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.273006    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 23:36:52.273134 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.273019    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Sep 04 23:36:52.273134 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.273073    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 23:36:52.273134 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.273094    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 23:36:52.273134 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.273108    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Sep 04 23:36:52.274393 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.274377    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.274454 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.274390    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/80e4b2c46b840ef3d591cbf9e0ff9c4c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"80e4b2c46b840ef3d591cbf9e0ff9c4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.274454 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.274405    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 23:36:52.274454 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.274421    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80e4b2c46b840ef3d591cbf9e0ff9c4c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"80e4b2c46b840ef3d591cbf9e0ff9c4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.274585 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.274456    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50d0ca4f5b95b0a1971e1cd22b27665d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-127.ec2.internal\" (UID: \"50d0ca4f5b95b0a1971e1cd22b27665d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.275087 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.275071    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 23:36:52.275177 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.275096    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 23:36:52.275177 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.275108    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeHasSufficientPID"
Sep 04 23:36:52.305441 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.305425    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-127.ec2.internal\" not found" node="ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.309733 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.309712    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-127.ec2.internal\" not found" node="ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.353216 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.353198    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:52.375310 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.375291    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/80e4b2c46b840ef3d591cbf9e0ff9c4c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"80e4b2c46b840ef3d591cbf9e0ff9c4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.375368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.375321    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80e4b2c46b840ef3d591cbf9e0ff9c4c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"80e4b2c46b840ef3d591cbf9e0ff9c4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.375368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.375338    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50d0ca4f5b95b0a1971e1cd22b27665d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-127.ec2.internal\" (UID: \"50d0ca4f5b95b0a1971e1cd22b27665d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.375436 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.375377    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50d0ca4f5b95b0a1971e1cd22b27665d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-127.ec2.internal\" (UID: \"50d0ca4f5b95b0a1971e1cd22b27665d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.375436 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.375400    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/80e4b2c46b840ef3d591cbf9e0ff9c4c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"80e4b2c46b840ef3d591cbf9e0ff9c4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.375436 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.375400    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80e4b2c46b840ef3d591cbf9e0ff9c4c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal\" (UID: \"80e4b2c46b840ef3d591cbf9e0ff9c4c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.453680 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.453651    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:52.554284 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.554227    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:52.607735 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.607716    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.612732 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:52.612694    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:52.654362 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.654317    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:52.754805 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.754778    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:52.855271 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.855220    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:52.955759 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:52.955738    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:53.017228 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:53.017204    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 04 23:36:53.017608 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:53.017335    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 04 23:36:53.056650 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:53.056625    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:53.082618 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:53.082576    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-03 23:31:52 +0000 UTC" deadline="2028-02-24 09:25:39.014361629 +0000 UTC"
Sep 04 23:36:53.082618 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:53.082610    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12897h48m45.931755046s"
Sep 04 23:36:53.157376 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:53.157317    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:53.208219 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:53.208193    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80e4b2c46b840ef3d591cbf9e0ff9c4c.slice/crio-5345d2c75490e2d3ae38bb33d3ded01bb2969e93eb512fc6588642adc41a824d WatchSource:0}: Error finding container 5345d2c75490e2d3ae38bb33d3ded01bb2969e93eb512fc6588642adc41a824d: Status 404 returned error can't find the container with id 5345d2c75490e2d3ae38bb33d3ded01bb2969e93eb512fc6588642adc41a824d
Sep 04 23:36:53.211154 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:53.211136    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 23:36:53.258176 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:53.258155    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:53.275214 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:53.275194    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d0ca4f5b95b0a1971e1cd22b27665d.slice/crio-ab757320bd3934f6d92244e5ccf51ebfd7a305a6aa0247e75b474a060e5515b1 WatchSource:0}: Error finding container ab757320bd3934f6d92244e5ccf51ebfd7a305a6aa0247e75b474a060e5515b1: Status 404 returned error can't find the container with id ab757320bd3934f6d92244e5ccf51ebfd7a305a6aa0247e75b474a060e5515b1
Sep 04 23:36:53.359197 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:53.359177    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-127.ec2.internal\" not found"
Sep 04 23:36:53.397602 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:53.397575    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 23:36:53.473659 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:53.473613    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:53.484151 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:53.484129    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 04 23:36:53.485750 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:53.485736    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal"
Sep 04 23:36:53.491599 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:53.491585    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 04 23:36:53.636525 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:53.636490    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 23:36:53.642781 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:53.642758    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 23:36:54.055443 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.055414    2564 apiserver.go:52] "Watching apiserver"
Sep 04 23:36:54.062438 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.062377    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 04 23:36:54.064524 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.064488    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-krlhk","kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal","openshift-cluster-node-tuning-operator/tuned-mm9dj","openshift-dns/node-resolver-2q7dw","openshift-image-registry/node-ca-4l98m","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal","openshift-multus/multus-additional-cni-plugins-b8z2h","openshift-multus/network-metrics-daemon-fk66m","kube-system/konnectivity-agent-29jcq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd","openshift-multus/multus-ff5c2","openshift-network-diagnostics/network-check-target-mrmc2","openshift-network-operator/iptables-alerter-fd9gw","openshift-ovn-kubernetes/ovnkube-node-lrgw5"]
Sep 04 23:36:54.067368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.067347    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.069713 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.069690    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-29jcq"
Sep 04 23:36:54.070380 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.070359    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 04 23:36:54.070492 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.070406    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 04 23:36:54.070570 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.070360    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 04 23:36:54.070700 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.070682    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mh9l8\""
Sep 04 23:36:54.070807 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.070793    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 04 23:36:54.070959 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.070940    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 04 23:36:54.072294 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.072177    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.073365 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.073345    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 04 23:36:54.073855 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.073830    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 04 23:36:54.074244 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.074225    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8ktjf\""
Sep 04 23:36:54.074432 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.074269    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 04 23:36:54.075052 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.074683    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2q7dw"
Sep 04 23:36:54.075052 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.074700    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 04 23:36:54.075187 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.075125    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 04 23:36:54.075187 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.075161    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 04 23:36:54.075403 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.075387    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 04 23:36:54.075403 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.075394    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m4rh4\""
Sep 04 23:36:54.075835 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.075651    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 04 23:36:54.075835 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.075721    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 04 23:36:54.077798 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.077523    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:36:54.077798 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.077632    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:36:54.079808 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.079787    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 04 23:36:54.080207 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.080186    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 04 23:36:54.080448 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.080430    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qjh8x\""
Sep 04 23:36:54.084705 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.084682    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.086071 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086048    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-run-netns\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086175 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086086    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-run-ovn\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086175 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086118    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086175 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086155    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f683ee03-698b-4e74-8fbb-427327ec0ff3-cnibin\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.086283 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086181    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f683ee03-698b-4e74-8fbb-427327ec0ff3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.086283 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/763c0843-c945-4233-9f1b-1da36b54e440-ovn-node-metrics-cert\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086283 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086231    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-run-systemd\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086283 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086256    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-run-ovn-kubernetes\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086459 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086282    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/763c0843-c945-4233-9f1b-1da36b54e440-ovnkube-config\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086459 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086306    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/763c0843-c945-4233-9f1b-1da36b54e440-env-overrides\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086459 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086330    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khth2\" (UniqueName: \"kubernetes.io/projected/763c0843-c945-4233-9f1b-1da36b54e440-kube-api-access-khth2\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086459 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-run-openvswitch\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086459 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086376    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-node-log\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086459 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086392    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4l98m"
Sep 04 23:36:54.086459 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086400    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/57fd64bb-51ba-4701-81bc-fbff66028fb2-hosts-file\") pod \"node-resolver-2q7dw\" (UID: \"57fd64bb-51ba-4701-81bc-fbff66028fb2\") " pod="openshift-dns/node-resolver-2q7dw"
Sep 04 23:36:54.086799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086497    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60746859-3df1-405d-88e6-4c686ac70181-konnectivity-ca\") pod \"konnectivity-agent-29jcq\" (UID: \"60746859-3df1-405d-88e6-4c686ac70181\") " pod="kube-system/konnectivity-agent-29jcq"
Sep 04 23:36:54.086799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086531    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-kubelet\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086559    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-cni-netd\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086584    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57fd64bb-51ba-4701-81bc-fbff66028fb2-tmp-dir\") pod \"node-resolver-2q7dw\" (UID: \"57fd64bb-51ba-4701-81bc-fbff66028fb2\") " pod="openshift-dns/node-resolver-2q7dw"
Sep 04 23:36:54.086799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086609    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f683ee03-698b-4e74-8fbb-427327ec0ff3-cni-binary-copy\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.086799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086638    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f683ee03-698b-4e74-8fbb-427327ec0ff3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.086799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086663    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60746859-3df1-405d-88e6-4c686ac70181-agent-certs\") pod \"konnectivity-agent-29jcq\" (UID: \"60746859-3df1-405d-88e6-4c686ac70181\") " pod="kube-system/konnectivity-agent-29jcq"
Sep 04 23:36:54.086799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086688    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-systemd-units\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086712    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-log-socket\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.086799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slbfr\" (UniqueName: \"kubernetes.io/projected/57fd64bb-51ba-4701-81bc-fbff66028fb2-kube-api-access-slbfr\") pod \"node-resolver-2q7dw\" (UID: \"57fd64bb-51ba-4701-81bc-fbff66028fb2\") " pod="openshift-dns/node-resolver-2q7dw"
Sep 04 23:36:54.086799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086761    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f683ee03-698b-4e74-8fbb-427327ec0ff3-system-cni-dir\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.086799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086787    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvv2w\" (UniqueName: \"kubernetes.io/projected/f683ee03-698b-4e74-8fbb-427327ec0ff3-kube-api-access-dvv2w\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.087309 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086811    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-slash\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.087309 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086839    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-cni-bin\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.087309 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086862    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/763c0843-c945-4233-9f1b-1da36b54e440-ovnkube-script-lib\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.087309 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086887    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f683ee03-698b-4e74-8fbb-427327ec0ff3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.087309 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086911    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-var-lib-openvswitch\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.087309 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.086950    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-etc-openvswitch\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.087509 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.087397    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 04 23:36:54.087599 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.087580    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 04 23:36:54.087707 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.087687    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zsntb\""
Sep 04 23:36:54.088739 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.088718    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fd9gw"
Sep 04 23:36:54.088911 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.088893    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 04 23:36:54.089253 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.089225    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-76xgw\""
Sep 04 23:36:54.089347 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.089263    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 04 23:36:54.089592 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.089572    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 04 23:36:54.092231 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.092213    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 04 23:36:54.092327 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.092280    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 04 23:36:54.092440 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.092423    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 04 23:36:54.092590 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.092571    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2fk7l\""
Sep 04 23:36:54.092763 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.092744    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.095021 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.094999    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:54.095125 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.095076    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:36:54.095303 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.095280    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 04 23:36:54.095499 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.095482    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 04 23:36:54.095499 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.095492    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bfhzv\""
Sep 04 23:36:54.095620 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.095558    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 04 23:36:54.098261 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.098242    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:36:54.098359 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.098313    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:36:54.100716 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.100697    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.103206 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.103184    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 04 23:36:54.103304 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.103225    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 04 23:36:54.103487 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.103463    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x74bp\""
Sep 04 23:36:54.107905 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.107877    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dblpm"
Sep 04 23:36:54.115531 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.115508    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dblpm"
Sep 04 23:36:54.173243 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.173189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" event={"ID":"50d0ca4f5b95b0a1971e1cd22b27665d","Type":"ContainerStarted","Data":"ab757320bd3934f6d92244e5ccf51ebfd7a305a6aa0247e75b474a060e5515b1"}
Sep 04 23:36:54.174733 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.174494    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 04 23:36:54.174836 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.174763    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" event={"ID":"80e4b2c46b840ef3d591cbf9e0ff9c4c","Type":"ContainerStarted","Data":"5345d2c75490e2d3ae38bb33d3ded01bb2969e93eb512fc6588642adc41a824d"}
Sep 04 23:36:54.187627 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187600    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m26m5\" (UniqueName: \"kubernetes.io/projected/ddba37cb-3134-411f-94de-20499e397628-kube-api-access-m26m5\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.187730 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187648    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvv2w\" (UniqueName: \"kubernetes.io/projected/f683ee03-698b-4e74-8fbb-427327ec0ff3-kube-api-access-dvv2w\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.187730 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187679    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-slash\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.187730 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187704    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-hostroot\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.187730 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee12b2d2-d830-4bcf-b1a3-1c6365afe41d-iptables-alerter-script\") pod \"iptables-alerter-fd9gw\" (UID: \"ee12b2d2-d830-4bcf-b1a3-1c6365afe41d\") " pod="openshift-network-operator/iptables-alerter-fd9gw"
Sep 04 23:36:54.187972 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187752    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:54.187972 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187778    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tbvl\" (UniqueName: \"kubernetes.io/projected/202d6558-e746-4911-9b99-f1b079343126-kube-api-access-8tbvl\") pod \"network-metrics-daemon-fk66m\" (UID: \"202d6558-e746-4911-9b99-f1b079343126\") " pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:36:54.187972 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187805    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f683ee03-698b-4e74-8fbb-427327ec0ff3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.187972 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187834    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vqvz\" (UniqueName: \"kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz\") pod \"network-check-target-mrmc2\" (UID: \"0b8a5cc2-85e1-4921-a572-71fa06de5611\") " pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:36:54.187972 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187859    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee12b2d2-d830-4bcf-b1a3-1c6365afe41d-host-slash\") pod \"iptables-alerter-fd9gw\" (UID: \"ee12b2d2-d830-4bcf-b1a3-1c6365afe41d\") " pod="openshift-network-operator/iptables-alerter-fd9gw"
Sep 04 23:36:54.187972 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187884    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.187972 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187909    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt59d\" (UniqueName: \"kubernetes.io/projected/05c55d44-a7e1-4399-bcca-00284a6490b0-kube-api-access-dt59d\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.187972 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187957    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddba37cb-3134-411f-94de-20499e397628-tmp\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.188365 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.187981    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs\") pod \"network-metrics-daemon-fk66m\" (UID: \"202d6558-e746-4911-9b99-f1b079343126\") " pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:36:54.188365 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188026    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.188365 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188054    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f683ee03-698b-4e74-8fbb-427327ec0ff3-cnibin\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.188365 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188085    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f683ee03-698b-4e74-8fbb-427327ec0ff3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.188365 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188184    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.188365 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188232    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f683ee03-698b-4e74-8fbb-427327ec0ff3-cnibin\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.188657 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-slash\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.188657 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188592    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-multus-cni-dir\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.188657 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188607    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f683ee03-698b-4e74-8fbb-427327ec0ff3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.188657 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188620    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-sysctl-d\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.188657 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-var-lib-kubelet\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.188902 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-run-systemd\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.188902 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188700    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-run-ovn-kubernetes\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.188902 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188724    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b94086d-38f4-4be0-857d-ae657478079d-host\") pod \"node-ca-4l98m\" (UID: \"2b94086d-38f4-4be0-857d-ae657478079d\") " pod="openshift-image-registry/node-ca-4l98m"
Sep 04 23:36:54.188902 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188749    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-socket-dir\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.188902 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188772    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f683ee03-698b-4e74-8fbb-427327ec0ff3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.188902 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188772    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-sys\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.188902 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188812    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-multus-socket-dir-parent\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.188902 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188815    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-run-systemd\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.188902 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188843    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4bwc\" (UniqueName: \"kubernetes.io/projected/ee12b2d2-d830-4bcf-b1a3-1c6365afe41d-kube-api-access-h4bwc\") pod \"iptables-alerter-fd9gw\" (UID: \"ee12b2d2-d830-4bcf-b1a3-1c6365afe41d\") " pod="openshift-network-operator/iptables-alerter-fd9gw"
Sep 04 23:36:54.188902 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188874    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-host\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.188902 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188878    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-run-ovn-kubernetes\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.188902 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188900    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-multus-daemon-config\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.188947    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60746859-3df1-405d-88e6-4c686ac70181-konnectivity-ca\") pod \"konnectivity-agent-29jcq\" (UID: \"60746859-3df1-405d-88e6-4c686ac70181\") " pod="kube-system/konnectivity-agent-29jcq"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189011    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57fd64bb-51ba-4701-81bc-fbff66028fb2-tmp-dir\") pod \"node-resolver-2q7dw\" (UID: \"57fd64bb-51ba-4701-81bc-fbff66028fb2\") " pod="openshift-dns/node-resolver-2q7dw"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189123    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-cnibin\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189141    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-run-k8s-cni-cncf-io\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189171    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddba37cb-3134-411f-94de-20499e397628-etc-tuned\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189188    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f683ee03-698b-4e74-8fbb-427327ec0ff3-cni-binary-copy\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189206    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f683ee03-698b-4e74-8fbb-427327ec0ff3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189222    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60746859-3df1-405d-88e6-4c686ac70181-agent-certs\") pod \"konnectivity-agent-29jcq\" (UID: \"60746859-3df1-405d-88e6-4c686ac70181\") " pod="kube-system/konnectivity-agent-29jcq"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189277    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-systemd-units\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189298    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-cni-binary-copy\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189313    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-var-lib-cni-bin\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189329    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8678835d-111a-4d74-84e3-372e11e75a50-kubelet-config\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189355    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f683ee03-698b-4e74-8fbb-427327ec0ff3-system-cni-dir\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189371    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-cni-bin\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189391    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/763c0843-c945-4233-9f1b-1da36b54e440-ovnkube-script-lib\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189414    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-etc-kubernetes\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.189577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189512    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60746859-3df1-405d-88e6-4c686ac70181-konnectivity-ca\") pod \"konnectivity-agent-29jcq\" (UID: \"60746859-3df1-405d-88e6-4c686ac70181\") " pod="kube-system/konnectivity-agent-29jcq"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.189942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/763c0843-c945-4233-9f1b-1da36b54e440-ovnkube-script-lib\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190005    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f683ee03-698b-4e74-8fbb-427327ec0ff3-cni-binary-copy\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-device-dir\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190083    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-var-lib-openvswitch\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-etc-openvswitch\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190136    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-sys-fs\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190192    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8678835d-111a-4d74-84e3-372e11e75a50-dbus\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190232    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-sysconfig\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190264    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-sysctl-conf\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190296    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-run-netns\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190321    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-run-ovn\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/763c0843-c945-4233-9f1b-1da36b54e440-ovn-node-metrics-cert\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190392    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr7ff\" (UniqueName: \"kubernetes.io/projected/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-kube-api-access-mr7ff\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190397    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-systemd-units\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190430    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-cni-bin\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190449    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f683ee03-698b-4e74-8fbb-427327ec0ff3-system-cni-dir\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.190505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190457    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/763c0843-c945-4233-9f1b-1da36b54e440-ovnkube-config\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190474    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/763c0843-c945-4233-9f1b-1da36b54e440-env-overrides\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190491    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khth2\" (UniqueName: \"kubernetes.io/projected/763c0843-c945-4233-9f1b-1da36b54e440-kube-api-access-khth2\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190510    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-run-netns\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190577    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-var-lib-cni-multus\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190613    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-var-lib-kubelet\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190632    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-registration-dir\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190656    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-run-openvswitch\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190673    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-node-log\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190688    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/57fd64bb-51ba-4701-81bc-fbff66028fb2-hosts-file\") pod \"node-resolver-2q7dw\" (UID: \"57fd64bb-51ba-4701-81bc-fbff66028fb2\") " pod="openshift-dns/node-resolver-2q7dw"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190704    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190724    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-modprobe-d\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190748    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-systemd\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190821    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf69w\" (UniqueName: \"kubernetes.io/projected/2b94086d-38f4-4be0-857d-ae657478079d-kube-api-access-nf69w\") pod \"node-ca-4l98m\" (UID: \"2b94086d-38f4-4be0-857d-ae657478079d\") " pod="openshift-image-registry/node-ca-4l98m"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190838    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-multus-conf-dir\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190854    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-run-multus-certs\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190868    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2b94086d-38f4-4be0-857d-ae657478079d-serviceca\") pod \"node-ca-4l98m\" (UID: \"2b94086d-38f4-4be0-857d-ae657478079d\") " pod="openshift-image-registry/node-ca-4l98m"
Sep 04 23:36:54.191259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190886    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-kubelet\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190902    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-cni-netd\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190929    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-system-cni-dir\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190947    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-lib-modules\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190993    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-log-socket\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191019    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slbfr\" (UniqueName: \"kubernetes.io/projected/57fd64bb-51ba-4701-81bc-fbff66028fb2-kube-api-access-slbfr\") pod \"node-resolver-2q7dw\" (UID: \"57fd64bb-51ba-4701-81bc-fbff66028fb2\") " pod="openshift-dns/node-resolver-2q7dw"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191037    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-kubernetes\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191053    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-run\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191137    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.190365    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f683ee03-698b-4e74-8fbb-427327ec0ff3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191474    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/763c0843-c945-4233-9f1b-1da36b54e440-ovnkube-config\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191502    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-run-netns\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191511    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-run-ovn\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191591    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-node-log\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191625    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-kubelet\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191629    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-run-openvswitch\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191675    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/57fd64bb-51ba-4701-81bc-fbff66028fb2-hosts-file\") pod \"node-resolver-2q7dw\" (UID: \"57fd64bb-51ba-4701-81bc-fbff66028fb2\") " pod="openshift-dns/node-resolver-2q7dw"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191687    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-log-socket\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191717    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-var-lib-openvswitch\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192831 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191743    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/763c0843-c945-4233-9f1b-1da36b54e440-env-overrides\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192831 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191752    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-etc-openvswitch\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192831 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191799    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/763c0843-c945-4233-9f1b-1da36b54e440-host-cni-netd\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.192831 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.191997    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57fd64bb-51ba-4701-81bc-fbff66028fb2-tmp-dir\") pod \"node-resolver-2q7dw\" (UID: \"57fd64bb-51ba-4701-81bc-fbff66028fb2\") " pod="openshift-dns/node-resolver-2q7dw"
Sep 04 23:36:54.195702 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.195667    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/763c0843-c945-4233-9f1b-1da36b54e440-ovn-node-metrics-cert\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.195806 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.195780    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60746859-3df1-405d-88e6-4c686ac70181-agent-certs\") pod \"konnectivity-agent-29jcq\" (UID: \"60746859-3df1-405d-88e6-4c686ac70181\") " pod="kube-system/konnectivity-agent-29jcq"
Sep 04 23:36:54.197515 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.197493    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvv2w\" (UniqueName: \"kubernetes.io/projected/f683ee03-698b-4e74-8fbb-427327ec0ff3-kube-api-access-dvv2w\") pod \"multus-additional-cni-plugins-b8z2h\" (UID: \"f683ee03-698b-4e74-8fbb-427327ec0ff3\") " pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.203694 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.203668    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slbfr\" (UniqueName: \"kubernetes.io/projected/57fd64bb-51ba-4701-81bc-fbff66028fb2-kube-api-access-slbfr\") pod \"node-resolver-2q7dw\" (UID: \"57fd64bb-51ba-4701-81bc-fbff66028fb2\") " pod="openshift-dns/node-resolver-2q7dw"
Sep 04 23:36:54.204109 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.204082    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khth2\" (UniqueName: \"kubernetes.io/projected/763c0843-c945-4233-9f1b-1da36b54e440-kube-api-access-khth2\") pod \"ovnkube-node-lrgw5\" (UID: \"763c0843-c945-4233-9f1b-1da36b54e440\") " pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.291892 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.291862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr7ff\" (UniqueName: \"kubernetes.io/projected/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-kube-api-access-mr7ff\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.291892 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.291895    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-run-netns\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.292108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.291927    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-var-lib-cni-multus\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.292108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.291945    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-var-lib-kubelet\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.292108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292002    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-var-lib-cni-multus\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.292108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292003    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-run-netns\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.292108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292025    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-registration-dir\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.292108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292064    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.292108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292079    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-registration-dir\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.292108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292090    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-modprobe-d\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-systemd\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292085    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-var-lib-kubelet\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292135    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf69w\" (UniqueName: \"kubernetes.io/projected/2b94086d-38f4-4be0-857d-ae657478079d-kube-api-access-nf69w\") pod \"node-ca-4l98m\" (UID: \"2b94086d-38f4-4be0-857d-ae657478079d\") " pod="openshift-image-registry/node-ca-4l98m"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292169    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-multus-conf-dir\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292187    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-modprobe-d\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-run-multus-certs\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292171    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-etc-selinux\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292203    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-multus-conf-dir\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292205    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-systemd\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292227    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2b94086d-38f4-4be0-857d-ae657478079d-serviceca\") pod \"node-ca-4l98m\" (UID: \"2b94086d-38f4-4be0-857d-ae657478079d\") " pod="openshift-image-registry/node-ca-4l98m"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-system-cni-dir\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292299    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-lib-modules\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292320    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-run-multus-certs\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292326    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-kubernetes\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292352    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-run\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292378    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m26m5\" (UniqueName: \"kubernetes.io/projected/ddba37cb-3134-411f-94de-20499e397628-kube-api-access-m26m5\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292397    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-system-cni-dir\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.292514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292408    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-hostroot\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292413    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-kubernetes\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292403    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-lib-modules\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292434    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee12b2d2-d830-4bcf-b1a3-1c6365afe41d-iptables-alerter-script\") pod \"iptables-alerter-fd9gw\" (UID: \"ee12b2d2-d830-4bcf-b1a3-1c6365afe41d\") " pod="openshift-network-operator/iptables-alerter-fd9gw"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292454    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-run\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292489    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tbvl\" (UniqueName: \"kubernetes.io/projected/202d6558-e746-4911-9b99-f1b079343126-kube-api-access-8tbvl\") pod \"network-metrics-daemon-fk66m\" (UID: \"202d6558-e746-4911-9b99-f1b079343126\") " pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292463    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-hostroot\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292515    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vqvz\" (UniqueName: \"kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz\") pod \"network-check-target-mrmc2\" (UID: \"0b8a5cc2-85e1-4921-a572-71fa06de5611\") " pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292537    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee12b2d2-d830-4bcf-b1a3-1c6365afe41d-host-slash\") pod \"iptables-alerter-fd9gw\" (UID: \"ee12b2d2-d830-4bcf-b1a3-1c6365afe41d\") " pod="openshift-network-operator/iptables-alerter-fd9gw"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.292542    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292562    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292588    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt59d\" (UniqueName: \"kubernetes.io/projected/05c55d44-a7e1-4399-bcca-00284a6490b0-kube-api-access-dt59d\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.292612    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret podName:8678835d-111a-4d74-84e3-372e11e75a50 nodeName:}" failed. No retries permitted until 2026-09-04 23:36:54.792591664 +0000 UTC m=+3.040592164 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret") pod "global-pull-secret-syncer-krlhk" (UID: "8678835d-111a-4d74-84e3-372e11e75a50") : object "kube-system"/"original-pull-secret" not registered
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292627    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2b94086d-38f4-4be0-857d-ae657478079d-serviceca\") pod \"node-ca-4l98m\" (UID: \"2b94086d-38f4-4be0-857d-ae657478079d\") " pod="openshift-image-registry/node-ca-4l98m"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292648    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee12b2d2-d830-4bcf-b1a3-1c6365afe41d-host-slash\") pod \"iptables-alerter-fd9gw\" (UID: \"ee12b2d2-d830-4bcf-b1a3-1c6365afe41d\") " pod="openshift-network-operator/iptables-alerter-fd9gw"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292697    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.293399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292761    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddba37cb-3134-411f-94de-20499e397628-tmp\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292782    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs\") pod \"network-metrics-daemon-fk66m\" (UID: \"202d6558-e746-4911-9b99-f1b079343126\") " pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292821    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-multus-cni-dir\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292840    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-sysctl-d\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292855    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-var-lib-kubelet\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.292887    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.292935    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs podName:202d6558-e746-4911-9b99-f1b079343126 nodeName:}" failed. No retries permitted until 2026-09-04 23:36:54.79292118 +0000 UTC m=+3.040921696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs") pod "network-metrics-daemon-fk66m" (UID: "202d6558-e746-4911-9b99-f1b079343126") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292939    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-multus-cni-dir\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292889    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b94086d-38f4-4be0-857d-ae657478079d-host\") pod \"node-ca-4l98m\" (UID: \"2b94086d-38f4-4be0-857d-ae657478079d\") " pod="openshift-image-registry/node-ca-4l98m"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292961    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee12b2d2-d830-4bcf-b1a3-1c6365afe41d-iptables-alerter-script\") pod \"iptables-alerter-fd9gw\" (UID: \"ee12b2d2-d830-4bcf-b1a3-1c6365afe41d\") " pod="openshift-network-operator/iptables-alerter-fd9gw"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292945    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2b94086d-38f4-4be0-857d-ae657478079d-host\") pod \"node-ca-4l98m\" (UID: \"2b94086d-38f4-4be0-857d-ae657478079d\") " pod="openshift-image-registry/node-ca-4l98m"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292985    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-var-lib-kubelet\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.292987    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-socket-dir\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293023    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-sys\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293040    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-multus-socket-dir-parent\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293061    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4bwc\" (UniqueName: \"kubernetes.io/projected/ee12b2d2-d830-4bcf-b1a3-1c6365afe41d-kube-api-access-h4bwc\") pod \"iptables-alerter-fd9gw\" (UID: \"ee12b2d2-d830-4bcf-b1a3-1c6365afe41d\") " pod="openshift-network-operator/iptables-alerter-fd9gw"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293070    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-sysctl-d\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.294364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293074    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-socket-dir\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293085    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-host\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293111    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-sys\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293122    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-multus-socket-dir-parent\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293125    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-multus-daemon-config\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293164    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-cnibin\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293169    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-host\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293189    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-run-k8s-cni-cncf-io\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293213    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-cnibin\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddba37cb-3134-411f-94de-20499e397628-etc-tuned\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293249    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-run-k8s-cni-cncf-io\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-cni-binary-copy\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293289    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-var-lib-cni-bin\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293315    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8678835d-111a-4d74-84e3-372e11e75a50-kubelet-config\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293345    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-etc-kubernetes\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293366    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-device-dir\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293382    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-sys-fs\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293396    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8678835d-111a-4d74-84e3-372e11e75a50-dbus\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:54.295220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293410    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-sysconfig\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.296170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293430    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-sysctl-conf\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.296170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293543    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-sysctl-conf\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.296170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293570    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-host-var-lib-cni-bin\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.296170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293597    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8678835d-111a-4d74-84e3-372e11e75a50-kubelet-config\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:54.296170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293621    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-etc-kubernetes\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.296170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293663    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-device-dir\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.296170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293672    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-cni-binary-copy\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.296170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293682    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8678835d-111a-4d74-84e3-372e11e75a50-dbus\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:54.296170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293708    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ddba37cb-3134-411f-94de-20499e397628-etc-sysconfig\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.296170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293733    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/05c55d44-a7e1-4399-bcca-00284a6490b0-sys-fs\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.296170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.293957    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-multus-daemon-config\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.296170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.296154    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ddba37cb-3134-411f-94de-20499e397628-etc-tuned\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.299658 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.299626    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddba37cb-3134-411f-94de-20499e397628-tmp\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.300154 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.300134    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 23:36:54.300301 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.300160    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 23:36:54.300301 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.300175    2564 projected.go:196] Error preparing data for projected volume kube-api-access-5vqvz for pod openshift-network-diagnostics/network-check-target-mrmc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:36:54.300301 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.300227    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz podName:0b8a5cc2-85e1-4921-a572-71fa06de5611 nodeName:}" failed. No retries permitted until 2026-09-04 23:36:54.800211262 +0000 UTC m=+3.048211774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5vqvz" (UniqueName: "kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz") pod "network-check-target-mrmc2" (UID: "0b8a5cc2-85e1-4921-a572-71fa06de5611") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:36:54.302560 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.302527    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tbvl\" (UniqueName: \"kubernetes.io/projected/202d6558-e746-4911-9b99-f1b079343126-kube-api-access-8tbvl\") pod \"network-metrics-daemon-fk66m\" (UID: \"202d6558-e746-4911-9b99-f1b079343126\") " pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:36:54.302945 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.302923    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr7ff\" (UniqueName: \"kubernetes.io/projected/4b6d3fb7-41a9-4ebd-aa16-7d70165952bb-kube-api-access-mr7ff\") pod \"multus-ff5c2\" (UID: \"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb\") " pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.303291 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.303271    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf69w\" (UniqueName: \"kubernetes.io/projected/2b94086d-38f4-4be0-857d-ae657478079d-kube-api-access-nf69w\") pod \"node-ca-4l98m\" (UID: \"2b94086d-38f4-4be0-857d-ae657478079d\") " pod="openshift-image-registry/node-ca-4l98m"
Sep 04 23:36:54.303390 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.303356    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4bwc\" (UniqueName: \"kubernetes.io/projected/ee12b2d2-d830-4bcf-b1a3-1c6365afe41d-kube-api-access-h4bwc\") pod \"iptables-alerter-fd9gw\" (UID: \"ee12b2d2-d830-4bcf-b1a3-1c6365afe41d\") " pod="openshift-network-operator/iptables-alerter-fd9gw"
Sep 04 23:36:54.303761 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.303736    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt59d\" (UniqueName: \"kubernetes.io/projected/05c55d44-a7e1-4399-bcca-00284a6490b0-kube-api-access-dt59d\") pod \"aws-ebs-csi-driver-node-jknzd\" (UID: \"05c55d44-a7e1-4399-bcca-00284a6490b0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.306057 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.306030    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m26m5\" (UniqueName: \"kubernetes.io/projected/ddba37cb-3134-411f-94de-20499e397628-kube-api-access-m26m5\") pod \"tuned-mm9dj\" (UID: \"ddba37cb-3134-411f-94de-20499e397628\") " pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.384851 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.384793    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-b8z2h"
Sep 04 23:36:54.393559 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.393538    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 23:36:54.395656 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.395637    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-29jcq"
Sep 04 23:36:54.404800 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.404777    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:36:54.411828 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.411804    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2q7dw"
Sep 04 23:36:54.423796 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.423778    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ff5c2"
Sep 04 23:36:54.430545 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.430522    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4l98m"
Sep 04 23:36:54.438193 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.438175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fd9gw"
Sep 04 23:36:54.444828 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.444812    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd"
Sep 04 23:36:54.451589 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.451571    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mm9dj"
Sep 04 23:36:54.797118 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.797039    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:54.797118 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.797102    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs\") pod \"network-metrics-daemon-fk66m\" (UID: \"202d6558-e746-4911-9b99-f1b079343126\") " pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:36:54.797324 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.797226    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 23:36:54.797324 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.797298    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret podName:8678835d-111a-4d74-84e3-372e11e75a50 nodeName:}" failed. No retries permitted until 2026-09-04 23:36:55.797279553 +0000 UTC m=+4.045280060 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret") pod "global-pull-secret-syncer-krlhk" (UID: "8678835d-111a-4d74-84e3-372e11e75a50") : object "kube-system"/"original-pull-secret" not registered
Sep 04 23:36:54.797324 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.797230    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:36:54.797459 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.797367    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs podName:202d6558-e746-4911-9b99-f1b079343126 nodeName:}" failed. No retries permitted until 2026-09-04 23:36:55.797353493 +0000 UTC m=+4.045354015 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs") pod "network-metrics-daemon-fk66m" (UID: "202d6558-e746-4911-9b99-f1b079343126") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:36:54.898194 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:54.898158    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vqvz\" (UniqueName: \"kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz\") pod \"network-check-target-mrmc2\" (UID: \"0b8a5cc2-85e1-4921-a572-71fa06de5611\") " pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:36:54.898358 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.898343    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 23:36:54.898412 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.898370    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 23:36:54.898412 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.898385    2564 projected.go:196] Error preparing data for projected volume kube-api-access-5vqvz for pod openshift-network-diagnostics/network-check-target-mrmc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:36:54.898485 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:54.898446    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz podName:0b8a5cc2-85e1-4921-a572-71fa06de5611 nodeName:}" failed. No retries permitted until 2026-09-04 23:36:55.898431078 +0000 UTC m=+4.146431578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vqvz" (UniqueName: "kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz") pod "network-check-target-mrmc2" (UID: "0b8a5cc2-85e1-4921-a572-71fa06de5611") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:36:55.119921 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:55.119796    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-03 23:31:54 +0000 UTC" deadline="2028-03-06 15:29:36.359596403 +0000 UTC"
Sep 04 23:36:55.119921 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:55.119854    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13167h52m41.239745351s"
Sep 04 23:36:55.243916 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:55.243885    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6d3fb7_41a9_4ebd_aa16_7d70165952bb.slice/crio-8b8f5a571ac7603e996e90fcfdc47d1946f19c8f6156e9bad390bfaa6da3bc32 WatchSource:0}: Error finding container 8b8f5a571ac7603e996e90fcfdc47d1946f19c8f6156e9bad390bfaa6da3bc32: Status 404 returned error can't find the container with id 8b8f5a571ac7603e996e90fcfdc47d1946f19c8f6156e9bad390bfaa6da3bc32
Sep 04 23:36:55.247156 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:55.246966    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60746859_3df1_405d_88e6_4c686ac70181.slice/crio-e61f9bbe0f4eab5f60be39f62e146d09ee88a419adafc7a656de621c8332a9a6 WatchSource:0}: Error finding container e61f9bbe0f4eab5f60be39f62e146d09ee88a419adafc7a656de621c8332a9a6: Status 404 returned error can't find the container with id e61f9bbe0f4eab5f60be39f62e146d09ee88a419adafc7a656de621c8332a9a6
Sep 04 23:36:55.251129 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:55.251105    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddba37cb_3134_411f_94de_20499e397628.slice/crio-e2a57b583985a9eda4088dd3ae7c74a4cf1d96efbe2137a61ebef36d7f3c4309 WatchSource:0}: Error finding container e2a57b583985a9eda4088dd3ae7c74a4cf1d96efbe2137a61ebef36d7f3c4309: Status 404 returned error can't find the container with id e2a57b583985a9eda4088dd3ae7c74a4cf1d96efbe2137a61ebef36d7f3c4309
Sep 04 23:36:55.252737 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:55.252668    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod763c0843_c945_4233_9f1b_1da36b54e440.slice/crio-a9f4ee2451de0bcac5a9e10730b7eeea0dc49510f2bf34611122259834869999 WatchSource:0}: Error finding container a9f4ee2451de0bcac5a9e10730b7eeea0dc49510f2bf34611122259834869999: Status 404 returned error can't find the container with id a9f4ee2451de0bcac5a9e10730b7eeea0dc49510f2bf34611122259834869999
Sep 04 23:36:55.255657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:55.255052    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee12b2d2_d830_4bcf_b1a3_1c6365afe41d.slice/crio-221ed4142ab1bb31f662751da54f4bda52a3d672377eaa0714ca618e2cc45c03 WatchSource:0}: Error finding container 221ed4142ab1bb31f662751da54f4bda52a3d672377eaa0714ca618e2cc45c03: Status 404 returned error can't find the container with id 221ed4142ab1bb31f662751da54f4bda52a3d672377eaa0714ca618e2cc45c03
Sep 04 23:36:55.255657 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:55.255416    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b94086d_38f4_4be0_857d_ae657478079d.slice/crio-33e51c1f07d364914d9e51a15bbac3588e254d4a050b97febf9463200aa343d2 WatchSource:0}: Error finding container 33e51c1f07d364914d9e51a15bbac3588e254d4a050b97febf9463200aa343d2: Status 404 returned error can't find the container with id 33e51c1f07d364914d9e51a15bbac3588e254d4a050b97febf9463200aa343d2
Sep 04 23:36:55.256353 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:36:55.256140    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05c55d44_a7e1_4399_bcca_00284a6490b0.slice/crio-248962d7466844ce5d43159692cc4a4558cafdb8ad0e09f3f65f03d84fbedc3a WatchSource:0}: Error finding container 248962d7466844ce5d43159692cc4a4558cafdb8ad0e09f3f65f03d84fbedc3a: Status 404 returned error can't find the container with id 248962d7466844ce5d43159692cc4a4558cafdb8ad0e09f3f65f03d84fbedc3a
Sep 04 23:36:55.807624 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:55.807287    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:55.807624 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:55.807349    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs\") pod \"network-metrics-daemon-fk66m\" (UID: \"202d6558-e746-4911-9b99-f1b079343126\") " pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:36:55.807624 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:55.807449    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 23:36:55.807624 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:55.807485    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:36:55.807624 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:55.807521    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret podName:8678835d-111a-4d74-84e3-372e11e75a50 nodeName:}" failed. No retries permitted until 2026-09-04 23:36:57.807498725 +0000 UTC m=+6.055499247 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret") pod "global-pull-secret-syncer-krlhk" (UID: "8678835d-111a-4d74-84e3-372e11e75a50") : object "kube-system"/"original-pull-secret" not registered
Sep 04 23:36:55.807624 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:55.807545    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs podName:202d6558-e746-4911-9b99-f1b079343126 nodeName:}" failed. No retries permitted until 2026-09-04 23:36:57.807528079 +0000 UTC m=+6.055528583 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs") pod "network-metrics-daemon-fk66m" (UID: "202d6558-e746-4911-9b99-f1b079343126") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:36:55.908742 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:55.908698    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vqvz\" (UniqueName: \"kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz\") pod \"network-check-target-mrmc2\" (UID: \"0b8a5cc2-85e1-4921-a572-71fa06de5611\") " pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:36:55.908934 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:55.908896    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 23:36:55.909016 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:55.908944    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 23:36:55.909016 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:55.908961    2564 projected.go:196] Error preparing data for projected volume kube-api-access-5vqvz for pod openshift-network-diagnostics/network-check-target-mrmc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:36:55.909132 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:55.909023    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz podName:0b8a5cc2-85e1-4921-a572-71fa06de5611 nodeName:}" failed. No retries permitted until 2026-09-04 23:36:57.909004019 +0000 UTC m=+6.157004536 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vqvz" (UniqueName: "kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz") pod "network-check-target-mrmc2" (UID: "0b8a5cc2-85e1-4921-a572-71fa06de5611") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:36:56.120691 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.120598    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-03 23:31:54 +0000 UTC" deadline="2028-02-10 02:01:06.436206566 +0000 UTC"
Sep 04 23:36:56.120691 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.120641    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12554h24m10.315571264s"
Sep 04 23:36:56.172363 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.171603    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:36:56.172363 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:56.171751    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:36:56.172363 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.172077    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:36:56.172363 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:56.172189    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:36:56.172363 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.172221    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:56.172363 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:56.172318    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:36:56.186111 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.186078    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8z2h" event={"ID":"f683ee03-698b-4e74-8fbb-427327ec0ff3","Type":"ContainerStarted","Data":"33dd0ffc9362f7ae5dead5594f6df96f8635fdf905a276cdb1cf77eaee4078e3"}
Sep 04 23:36:56.196822 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.196764    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" event={"ID":"50d0ca4f5b95b0a1971e1cd22b27665d","Type":"ContainerStarted","Data":"f2d13564e0320245ec36f91c1fa9231938221e23f5b08329b4209049241ba59a"}
Sep 04 23:36:56.202028 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.202003    2564 generic.go:356] "Generic (PLEG): container finished" podID="80e4b2c46b840ef3d591cbf9e0ff9c4c" containerID="3671da57c2dd4a0ed707b7860960a5d6e7185044e3018a46ddc90493a7cf29bd" exitCode=0
Sep 04 23:36:56.202127 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.202092    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" event={"ID":"80e4b2c46b840ef3d591cbf9e0ff9c4c","Type":"ContainerDied","Data":"3671da57c2dd4a0ed707b7860960a5d6e7185044e3018a46ddc90493a7cf29bd"}
Sep 04 23:36:56.230603 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.228219    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2q7dw" event={"ID":"57fd64bb-51ba-4701-81bc-fbff66028fb2","Type":"ContainerStarted","Data":"39c1a53e8ab8759e1f16b8f1cb7522269f3adbdda43cb648e28b951423ba33da"}
Sep 04 23:36:56.240946 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.240914    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fd9gw" event={"ID":"ee12b2d2-d830-4bcf-b1a3-1c6365afe41d","Type":"ContainerStarted","Data":"221ed4142ab1bb31f662751da54f4bda52a3d672377eaa0714ca618e2cc45c03"}
Sep 04 23:36:56.242317 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.242264    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-127.ec2.internal" podStartSLOduration=2.242244695 podStartE2EDuration="2.242244695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:36:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:36:56.211774458 +0000 UTC m=+4.459774977" watchObservedRunningTime="2026-09-04 23:36:56.242244695 +0000 UTC m=+4.490245216"
Sep 04 23:36:56.248889 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.248850    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4l98m" event={"ID":"2b94086d-38f4-4be0-857d-ae657478079d","Type":"ContainerStarted","Data":"33e51c1f07d364914d9e51a15bbac3588e254d4a050b97febf9463200aa343d2"}
Sep 04 23:36:56.255911 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.255885    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mm9dj" event={"ID":"ddba37cb-3134-411f-94de-20499e397628","Type":"ContainerStarted","Data":"e2a57b583985a9eda4088dd3ae7c74a4cf1d96efbe2137a61ebef36d7f3c4309"}
Sep 04 23:36:56.260246 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.260218    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-29jcq" event={"ID":"60746859-3df1-405d-88e6-4c686ac70181","Type":"ContainerStarted","Data":"e61f9bbe0f4eab5f60be39f62e146d09ee88a419adafc7a656de621c8332a9a6"}
Sep 04 23:36:56.271521 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.271492    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ff5c2" event={"ID":"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb","Type":"ContainerStarted","Data":"8b8f5a571ac7603e996e90fcfdc47d1946f19c8f6156e9bad390bfaa6da3bc32"}
Sep 04 23:36:56.280528 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.280463    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd" event={"ID":"05c55d44-a7e1-4399-bcca-00284a6490b0","Type":"ContainerStarted","Data":"248962d7466844ce5d43159692cc4a4558cafdb8ad0e09f3f65f03d84fbedc3a"}
Sep 04 23:36:56.290267 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:56.290243    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5" event={"ID":"763c0843-c945-4233-9f1b-1da36b54e440","Type":"ContainerStarted","Data":"a9f4ee2451de0bcac5a9e10730b7eeea0dc49510f2bf34611122259834869999"}
Sep 04 23:36:57.307361 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:57.305035    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" event={"ID":"80e4b2c46b840ef3d591cbf9e0ff9c4c","Type":"ContainerStarted","Data":"8d1b02c7df0d8c5539de5a2ca049661aff06b142d68586233fc6507c102a3a92"}
Sep 04 23:36:57.825498 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:57.825459    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:57.825686 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:57.825529    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs\") pod \"network-metrics-daemon-fk66m\" (UID: \"202d6558-e746-4911-9b99-f1b079343126\") " pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:36:57.825751 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:57.825698    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:36:57.825809 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:57.825762    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs podName:202d6558-e746-4911-9b99-f1b079343126 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:01.825741881 +0000 UTC m=+10.073742380 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs") pod "network-metrics-daemon-fk66m" (UID: "202d6558-e746-4911-9b99-f1b079343126") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:36:57.826216 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:57.826195    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 23:36:57.826327 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:57.826256    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret podName:8678835d-111a-4d74-84e3-372e11e75a50 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:01.826239614 +0000 UTC m=+10.074240113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret") pod "global-pull-secret-syncer-krlhk" (UID: "8678835d-111a-4d74-84e3-372e11e75a50") : object "kube-system"/"original-pull-secret" not registered
Sep 04 23:36:57.926027 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:57.925989    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vqvz\" (UniqueName: \"kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz\") pod \"network-check-target-mrmc2\" (UID: \"0b8a5cc2-85e1-4921-a572-71fa06de5611\") " pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:36:57.926207 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:57.926179    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 23:36:57.926207 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:57.926204    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 23:36:57.926328 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:57.926218    2564 projected.go:196] Error preparing data for projected volume kube-api-access-5vqvz for pod openshift-network-diagnostics/network-check-target-mrmc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:36:57.926328 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:57.926276    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz podName:0b8a5cc2-85e1-4921-a572-71fa06de5611 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:01.926258267 +0000 UTC m=+10.174258766 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vqvz" (UniqueName: "kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz") pod "network-check-target-mrmc2" (UID: "0b8a5cc2-85e1-4921-a572-71fa06de5611") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:36:58.171035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:58.170937    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:36:58.171234 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:58.171094    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:36:58.171524 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:58.171504    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:36:58.171844 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:58.171614    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:36:58.171844 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:36:58.171694    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:36:58.171844 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:36:58.171790    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:00.168806 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:00.168767    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:00.169373 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:00.168767    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:00.169373 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:00.168772    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:00.169373 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:00.168937    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:00.169373 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:00.168999    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:00.169373 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:00.169094    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:01.861482 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:01.861441    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:01.861918 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:01.861518    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs\") pod \"network-metrics-daemon-fk66m\" (UID: \"202d6558-e746-4911-9b99-f1b079343126\") " pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:01.861918 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:01.861677    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:37:01.861918 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:01.861745    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs podName:202d6558-e746-4911-9b99-f1b079343126 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:09.861722334 +0000 UTC m=+18.109722881 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs") pod "network-metrics-daemon-fk66m" (UID: "202d6558-e746-4911-9b99-f1b079343126") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:37:01.862247 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:01.862224    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 23:37:01.862348 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:01.862292    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret podName:8678835d-111a-4d74-84e3-372e11e75a50 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:09.862274232 +0000 UTC m=+18.110274746 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret") pod "global-pull-secret-syncer-krlhk" (UID: "8678835d-111a-4d74-84e3-372e11e75a50") : object "kube-system"/"original-pull-secret" not registered
Sep 04 23:37:01.961919 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:01.961885    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vqvz\" (UniqueName: \"kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz\") pod \"network-check-target-mrmc2\" (UID: \"0b8a5cc2-85e1-4921-a572-71fa06de5611\") " pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:01.962116 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:01.962095    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 23:37:01.962116 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:01.962118    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 23:37:01.962310 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:01.962130    2564 projected.go:196] Error preparing data for projected volume kube-api-access-5vqvz for pod openshift-network-diagnostics/network-check-target-mrmc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:37:01.962310 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:01.962189    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz podName:0b8a5cc2-85e1-4921-a572-71fa06de5611 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:09.962170459 +0000 UTC m=+18.210170972 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vqvz" (UniqueName: "kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz") pod "network-check-target-mrmc2" (UID: "0b8a5cc2-85e1-4921-a572-71fa06de5611") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:37:02.169731 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:02.169657    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:02.169894 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:02.169781    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:02.169894 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:02.169778    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:02.169894 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:02.169869    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:02.170101 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:02.169928    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:02.170204 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:02.170098    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:04.170037 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:04.170006    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:04.170509 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:04.170200    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:04.170509 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:04.170087    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:04.170509 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:04.170355    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:04.170509 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:04.170395    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:04.170509 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:04.170457    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:06.169513 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:06.169093    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:06.169513 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:06.169139    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:06.169513 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:06.169137    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:06.169513 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:06.169207    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:06.169513 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:06.169355    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:06.169513 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:06.169481    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:08.168185 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:08.168156    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:08.168185 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:08.168171    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:08.168774 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:08.168194    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:08.168774 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:08.168282    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:08.168774 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:08.168416    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:08.168774 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:08.168515    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:09.915001 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:09.914951    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:09.915421 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:09.915030    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs\") pod \"network-metrics-daemon-fk66m\" (UID: \"202d6558-e746-4911-9b99-f1b079343126\") " pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:09.915421 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:09.915123    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 23:37:09.915421 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:09.915152    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:37:09.915421 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:09.915195    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret podName:8678835d-111a-4d74-84e3-372e11e75a50 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:25.91517262 +0000 UTC m=+34.163173132 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret") pod "global-pull-secret-syncer-krlhk" (UID: "8678835d-111a-4d74-84e3-372e11e75a50") : object "kube-system"/"original-pull-secret" not registered
Sep 04 23:37:09.915421 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:09.915213    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs podName:202d6558-e746-4911-9b99-f1b079343126 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:25.915205732 +0000 UTC m=+34.163206226 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs") pod "network-metrics-daemon-fk66m" (UID: "202d6558-e746-4911-9b99-f1b079343126") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:37:10.015403 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:10.015329    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vqvz\" (UniqueName: \"kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz\") pod \"network-check-target-mrmc2\" (UID: \"0b8a5cc2-85e1-4921-a572-71fa06de5611\") " pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:10.015528 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:10.015512    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 23:37:10.015567 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:10.015533    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 23:37:10.015567 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:10.015542    2564 projected.go:196] Error preparing data for projected volume kube-api-access-5vqvz for pod openshift-network-diagnostics/network-check-target-mrmc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:37:10.015645 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:10.015588    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz podName:0b8a5cc2-85e1-4921-a572-71fa06de5611 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:26.015575163 +0000 UTC m=+34.263575657 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vqvz" (UniqueName: "kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz") pod "network-check-target-mrmc2" (UID: "0b8a5cc2-85e1-4921-a572-71fa06de5611") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:37:10.168808 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:10.168780    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:10.169015 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:10.168781    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:10.169015 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:10.168881    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:10.169015 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:10.168987    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:10.169015 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:10.168789    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:10.169251 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:10.169102    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:12.169274 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:12.169244    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:12.169650 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:12.169336    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:12.169650 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:12.169438    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:12.169650 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:12.169529    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:12.169650 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:12.169579    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:12.169793 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:12.169661    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:14.168660 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.168457    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:14.169382 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.168476    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:14.169382 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:14.168760    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:14.169382 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.168484    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:14.169382 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:14.168834    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:14.169382 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:14.168923    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:14.342500 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.342429    2564 generic.go:356] "Generic (PLEG): container finished" podID="f683ee03-698b-4e74-8fbb-427327ec0ff3" containerID="8ddcf46bbd3f44f062ff2863ee38c69cf8e072b68fc3574d31689233552ead8c" exitCode=0
Sep 04 23:37:14.342621 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.342515    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8z2h" event={"ID":"f683ee03-698b-4e74-8fbb-427327ec0ff3","Type":"ContainerDied","Data":"8ddcf46bbd3f44f062ff2863ee38c69cf8e072b68fc3574d31689233552ead8c"}
Sep 04 23:37:14.344722 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.344699    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2q7dw" event={"ID":"57fd64bb-51ba-4701-81bc-fbff66028fb2","Type":"ContainerStarted","Data":"a94f53b643e9e8e104274b15c4c10a5a2165c544414871342f3f38cd000a8067"}
Sep 04 23:37:14.346402 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.346341    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4l98m" event={"ID":"2b94086d-38f4-4be0-857d-ae657478079d","Type":"ContainerStarted","Data":"3113f022b6a14690dddb20de2440c96b253a8c21d743c8c5acfc7e7e25586e61"}
Sep 04 23:37:14.348745 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.348129    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mm9dj" event={"ID":"ddba37cb-3134-411f-94de-20499e397628","Type":"ContainerStarted","Data":"93b7cdd37b71d1e41415546b5a0e852bcd8ca46e578f76fc36a88369fd57553a"}
Sep 04 23:37:14.349782 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.349760    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-29jcq" event={"ID":"60746859-3df1-405d-88e6-4c686ac70181","Type":"ContainerStarted","Data":"fe96043ed8cae813813906bb6294f9e0cce6b47d15c838523bd2d0cce2f6f10e"}
Sep 04 23:37:14.351346 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.351325    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ff5c2" event={"ID":"4b6d3fb7-41a9-4ebd-aa16-7d70165952bb","Type":"ContainerStarted","Data":"d3eef036db27b3858ef820704be8c61104b3c007a75304f827e644d3d01c7d26"}
Sep 04 23:37:14.353017 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.352993    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd" event={"ID":"05c55d44-a7e1-4399-bcca-00284a6490b0","Type":"ContainerStarted","Data":"6fdab00a596a9493fdd51b02080afa7df4621ac22268a79bb61a322d1524fd7f"}
Sep 04 23:37:14.356469 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.356447    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5" event={"ID":"763c0843-c945-4233-9f1b-1da36b54e440","Type":"ContainerStarted","Data":"d4f63510a46c9679bf1e534eaf4c2cde71cd377c2d243bffb63d7ae698c297b4"}
Sep 04 23:37:14.356577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.356521    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5" event={"ID":"763c0843-c945-4233-9f1b-1da36b54e440","Type":"ContainerStarted","Data":"06a59585293b59ddeb36c79791b474c5394a47377fb2b5c3475ca7de283dc738"}
Sep 04 23:37:14.356577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.356568    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5" event={"ID":"763c0843-c945-4233-9f1b-1da36b54e440","Type":"ContainerStarted","Data":"571067a560fc119d3cfb1b348d7426e240e31e50f740dde6129d87374d1b3aad"}
Sep 04 23:37:14.356684 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.356617    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5" event={"ID":"763c0843-c945-4233-9f1b-1da36b54e440","Type":"ContainerStarted","Data":"b612f70f155f0bb042802cf275d9cd87f8bcc17b29c430f59ca05122eaf88c25"}
Sep 04 23:37:14.356684 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.356659    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5" event={"ID":"763c0843-c945-4233-9f1b-1da36b54e440","Type":"ContainerStarted","Data":"ed5f6a5035c4fcbb6f63ce1bb3721a0d09c5fe62eb95863558ab6accf51ae978"}
Sep 04 23:37:14.356789 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.356687    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5" event={"ID":"763c0843-c945-4233-9f1b-1da36b54e440","Type":"ContainerStarted","Data":"4f9d1eb80c49fec03aef7a3c697465ed6fa90fa159a89f992fd0112a557bfe69"}
Sep 04 23:37:14.363991 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.363945    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-127.ec2.internal" podStartSLOduration=20.363929445 podStartE2EDuration="20.363929445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:36:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:36:57.320918139 +0000 UTC m=+5.568918660" watchObservedRunningTime="2026-09-04 23:37:14.363929445 +0000 UTC m=+22.611929962"
Sep 04 23:37:14.396008 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.395964    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-29jcq" podStartSLOduration=4.207137051 podStartE2EDuration="22.39595142s" totalImagesPullingTime="18.188814369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:14.377663508 +0000 UTC m=+22.625664042" watchObservedRunningTime="2026-09-04 23:37:14.39595142 +0000 UTC m=+22.643951935"
Sep 04 23:37:14.396173 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.396144    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ff5c2" podStartSLOduration=4.12339119 podStartE2EDuration="22.396136652s" totalImagesPullingTime="18.272745462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:14.395406165 +0000 UTC m=+22.643406680" watchObservedRunningTime="2026-09-04 23:37:14.396136652 +0000 UTC m=+22.644137190"
Sep 04 23:37:14.410909 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.410858    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4l98m" podStartSLOduration=3.204738292 podStartE2EDuration="21.410846431s" totalImagesPullingTime="18.206108139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:14.410492992 +0000 UTC m=+22.658493540" watchObservedRunningTime="2026-09-04 23:37:14.410846431 +0000 UTC m=+22.658846948"
Sep 04 23:37:14.424438 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:14.424400    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2q7dw" podStartSLOduration=4.216068018 podStartE2EDuration="22.424387877s" totalImagesPullingTime="18.208319859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:14.424224411 +0000 UTC m=+22.672224926" watchObservedRunningTime="2026-09-04 23:37:14.424387877 +0000 UTC m=+22.672388393"
Sep 04 23:37:15.182665 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:15.182517    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 04 23:37:15.358996 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:15.358918    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:15.359883 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:15.359860    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fd9gw" event={"ID":"ee12b2d2-d830-4bcf-b1a3-1c6365afe41d","Type":"ContainerStarted","Data":"c923f78d0fde19c9f216ec4f33087fb95f1c99fac9d1ac1b69d29bd1f9070d72"}
Sep 04 23:37:15.362564 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:15.362534    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd" event={"ID":"05c55d44-a7e1-4399-bcca-00284a6490b0","Type":"ContainerStarted","Data":"7f8ae26a0a597c385d5b101456155d3f9d909c689a950f192945225badb9606e"}
Sep 04 23:37:15.373146 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:15.373103    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fd9gw" podStartSLOduration=4.165856976 podStartE2EDuration="22.373085805s" totalImagesPullingTime="18.207228829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:15.372944218 +0000 UTC m=+23.620944760" watchObservedRunningTime="2026-09-04 23:37:15.373085805 +0000 UTC m=+23.621086322"
Sep 04 23:37:15.373415 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:15.373383    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mm9dj" podStartSLOduration=4.162420994 podStartE2EDuration="22.373375941s" totalImagesPullingTime="18.210954947s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:14.439881581 +0000 UTC m=+22.687882097" watchObservedRunningTime="2026-09-04 23:37:15.373375941 +0000 UTC m=+23.621376458"
Sep 04 23:37:15.436168 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:15.436124    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:15.536241 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:15.536195    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:15.690547 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:15.690485    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:15.877530 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:15.877501    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:15.977140 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:15.977063    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:16.077575 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:16.077544    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:16.144010 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:16.143877    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-04T23:37:15.182661076Z","UUID":"8bb53f5a-bca6-4ae5-9051-e325dbcf5390","Handler":null,"Name":"","Endpoint":""}
Sep 04 23:37:16.145924 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:16.145638    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 04 23:37:16.145924 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:16.145673    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 04 23:37:16.168493 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:16.168468    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:16.168630 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:16.168471    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:16.168630 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:16.168598    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:16.168630 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:16.168598    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:16.168762 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:16.168667    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:16.168799 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:16.168782    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:16.369741 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:16.369703    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5" event={"ID":"763c0843-c945-4233-9f1b-1da36b54e440","Type":"ContainerStarted","Data":"12e651142149c3f6e541cd3c5e557e505f0cb3fafd1bd480a0ae91008692af86"}
Sep 04 23:37:16.440076 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:16.440043    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:16.540936 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:16.540848    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:17.373969 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:17.373932    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd" event={"ID":"05c55d44-a7e1-4399-bcca-00284a6490b0","Type":"ContainerStarted","Data":"2f29e521d9d21cc26d5eda0d268ab6efc9245f6d5879fae878c009f05a8ef6be"}
Sep 04 23:37:17.391213 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:17.391164    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jknzd" podStartSLOduration=3.753081313 podStartE2EDuration="24.391150272s" totalImagesPullingTime="20.638068959s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:36:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:17.390266436 +0000 UTC m=+25.638266975" watchObservedRunningTime="2026-09-04 23:37:17.391150272 +0000 UTC m=+25.639150790"
Sep 04 23:37:17.439629 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:17.439593    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:17.540822 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:17.540773    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:17.849461 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:17.849381    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:18.035455 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:18.035420    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:18.168230 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:18.168166    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:18.168389 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:18.168166    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:18.168389 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:18.168285    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:18.168389 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:18.168295    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:18.168389 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:18.168366    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:18.168558 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:18.168437    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:18.235175 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:18.235125    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:18.431080 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:18.431009    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:18.857171 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:18.857100    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-29jcq"
Sep 04 23:37:18.857846 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:18.857817    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-29jcq"
Sep 04 23:37:19.033607 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:19.033579    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:19.296711 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:19.296595    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:19.382212 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:19.382183    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5" event={"ID":"763c0843-c945-4233-9f1b-1da36b54e440","Type":"ContainerStarted","Data":"12f1f79696f5f2a89a243655da7c9db09453d2c20a289575c12c9a95a56221c5"}
Sep 04 23:37:19.382374 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:19.382350    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-29jcq"
Sep 04 23:37:19.382969 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:19.382951    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-29jcq"
Sep 04 23:37:19.410911 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:19.410859    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5" podStartSLOduration=9.147388773 podStartE2EDuration="27.410848117s" totalImagesPullingTime="18.263459344s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:19.409413855 +0000 UTC m=+27.657414411" watchObservedRunningTime="2026-09-04 23:37:19.410848117 +0000 UTC m=+27.658848634"
Sep 04 23:37:19.441119 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:19.441094    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:19.871251 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:19.871183    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:19.914689 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:19.914666    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:37:20.037000 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.036971    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:20.139207 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.139154    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:20.169027 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.169002    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:20.169154 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.169137    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:20.169216 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:20.169147    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:20.169216 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.169167    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:20.169317 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:20.169244    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:20.169367 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:20.169320    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:20.238556 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.238529    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:20.386205 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.386184    2564 generic.go:356] "Generic (PLEG): container finished" podID="f683ee03-698b-4e74-8fbb-427327ec0ff3" containerID="7a4d52ec40d21b8ec8f988cdbe96ac4c9fce8e434b51cc252b7c305a479333b7" exitCode=0
Sep 04 23:37:20.386314 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.386276    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8z2h" event={"ID":"f683ee03-698b-4e74-8fbb-427327ec0ff3","Type":"ContainerDied","Data":"7a4d52ec40d21b8ec8f988cdbe96ac4c9fce8e434b51cc252b7c305a479333b7"}
Sep 04 23:37:20.387317 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.386859    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:37:20.387317 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.386945    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:37:20.402089 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.402044    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:37:20.402193 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.402179    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:37:20.634786 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.634761    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:20.848039 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:20.848022    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:21.127101 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:21.127019    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:21.235079 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:21.235055    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:21.638245 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:21.638217    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:21.831743 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:21.831655    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:22.169396 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:22.169367    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:22.169503 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:22.169493    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:22.169582 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:22.169548    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:22.169582 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:22.169585    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:22.169746 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:22.169611    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:22.169746 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:22.169688    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:22.392700 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:22.392671    2564 generic.go:356] "Generic (PLEG): container finished" podID="f683ee03-698b-4e74-8fbb-427327ec0ff3" containerID="4554587d43e224d6bd94879778761b118d004145e655422398ff27faedc5d9fa" exitCode=0
Sep 04 23:37:22.392816 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:22.392757    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8z2h" event={"ID":"f683ee03-698b-4e74-8fbb-427327ec0ff3","Type":"ContainerDied","Data":"4554587d43e224d6bd94879778761b118d004145e655422398ff27faedc5d9fa"}
Sep 04 23:37:22.442024 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:22.441819    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:22.836472 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:22.836417    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:23.043211 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:23.043177    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:23.440514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:23.440487    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:23.837379 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:23.837353    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:24.168401 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:24.168373    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:24.168401 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:24.168389    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:24.168618 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:24.168461    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:24.168618 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:24.168502    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:24.168618 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:24.168582    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:24.168754 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:24.168669    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:24.399636 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:24.399599    2564 generic.go:356] "Generic (PLEG): container finished" podID="f683ee03-698b-4e74-8fbb-427327ec0ff3" containerID="de1e144a79da80117b975a10e44880b15198802694879f84980599c052f2fde6" exitCode=0
Sep 04 23:37:24.399812 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:24.399650    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8z2h" event={"ID":"f683ee03-698b-4e74-8fbb-427327ec0ff3","Type":"ContainerDied","Data":"de1e144a79da80117b975a10e44880b15198802694879f84980599c052f2fde6"}
Sep 04 23:37:24.434181 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:24.434106    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:24.834178 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:24.834080    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:25.248373 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:25.248340    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:25.436081 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:25.435998    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:25.635627 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:25.635603    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:25.938410 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:25.938337    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs\") pod \"network-metrics-daemon-fk66m\" (UID: \"202d6558-e746-4911-9b99-f1b079343126\") " pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:25.938410 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:25.938394    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:25.938587 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:25.938504    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 23:37:25.938587 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:25.938504    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:37:25.938587 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:25.938566    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret podName:8678835d-111a-4d74-84e3-372e11e75a50 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:57.938548522 +0000 UTC m=+66.186549023 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret") pod "global-pull-secret-syncer-krlhk" (UID: "8678835d-111a-4d74-84e3-372e11e75a50") : object "kube-system"/"original-pull-secret" not registered
Sep 04 23:37:25.938587 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:25.938584    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs podName:202d6558-e746-4911-9b99-f1b079343126 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:57.938574952 +0000 UTC m=+66.186575459 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs") pod "network-metrics-daemon-fk66m" (UID: "202d6558-e746-4911-9b99-f1b079343126") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 23:37:26.039456 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:26.039424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vqvz\" (UniqueName: \"kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz\") pod \"network-check-target-mrmc2\" (UID: \"0b8a5cc2-85e1-4921-a572-71fa06de5611\") " pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:26.039603 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:26.039543    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 23:37:26.039603 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:26.039560    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 23:37:26.039603 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:26.039569    2564 projected.go:196] Error preparing data for projected volume kube-api-access-5vqvz for pod openshift-network-diagnostics/network-check-target-mrmc2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:37:26.039713 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:26.039619    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz podName:0b8a5cc2-85e1-4921-a572-71fa06de5611 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:58.03960291 +0000 UTC m=+66.287603405 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vqvz" (UniqueName: "kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz") pod "network-check-target-mrmc2" (UID: "0b8a5cc2-85e1-4921-a572-71fa06de5611") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 23:37:26.168649 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:26.168619    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:26.168802 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:26.168619    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:26.168802 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:26.168717    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:26.168884 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:26.168826    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:26.168884 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:26.168619    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:26.169000 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:26.168953    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:28.168256 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:28.168209    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:28.168256 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:28.168234    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:28.168748 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:28.168339    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:28.168748 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:28.168349    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:28.168841 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:28.168760    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:28.169178 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:28.169140    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:29.414682 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:29.414657    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8z2h" event={"ID":"f683ee03-698b-4e74-8fbb-427327ec0ff3","Type":"ContainerStarted","Data":"066a02ded8277e3547315235e728b7cefa872d9dfbcfa941e8926e9f252001fa"}
Sep 04 23:37:30.168476 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:30.168444    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:30.168610 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:30.168541    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:30.168610 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:30.168454    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:30.168680 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:30.168622    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:30.168680 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:30.168455    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:30.168790 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:30.168762    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:30.356217 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:30.356173    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:30.419938 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:30.419846    2564 generic.go:356] "Generic (PLEG): container finished" podID="f683ee03-698b-4e74-8fbb-427327ec0ff3" containerID="066a02ded8277e3547315235e728b7cefa872d9dfbcfa941e8926e9f252001fa" exitCode=0
Sep 04 23:37:30.420286 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:30.419937    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8z2h" event={"ID":"f683ee03-698b-4e74-8fbb-427327ec0ff3","Type":"ContainerDied","Data":"066a02ded8277e3547315235e728b7cefa872d9dfbcfa941e8926e9f252001fa"}
Sep 04 23:37:30.457660 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:30.457629    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:30.654484 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:30.654461    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:31.026988 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:31.026951    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:31.128111 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:31.128081    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:31.243242 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:31.243196    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:31.424632 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:31.424603    2564 generic.go:356] "Generic (PLEG): container finished" podID="f683ee03-698b-4e74-8fbb-427327ec0ff3" containerID="45bcb817cc7a1a95d31b3fb85fdb510f11cd734015e6feb49bbe118dbac34703" exitCode=0
Sep 04 23:37:31.424632 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:31.424638    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8z2h" event={"ID":"f683ee03-698b-4e74-8fbb-427327ec0ff3","Type":"ContainerDied","Data":"45bcb817cc7a1a95d31b3fb85fdb510f11cd734015e6feb49bbe118dbac34703"}
Sep 04 23:37:31.425926 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:31.425886    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:32.102970 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:32.102942    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:32.169088 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:32.169054    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:32.169271 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:32.169148    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:32.169271 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:32.169163    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:32.169271 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:32.169220    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:32.169435 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:32.169293    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:32.169435 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:32.169329    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:32.236403 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:32.236374    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:32.326183 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:32.326142    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:32.427486 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:32.427451    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:32.431429 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:32.431400    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-b8z2h" event={"ID":"f683ee03-698b-4e74-8fbb-427327ec0ff3","Type":"ContainerStarted","Data":"462e4a6908202768b2289b72b8d0da0e3eafd5c554360f760d693824ea469b05"}
Sep 04 23:37:32.521877 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:32.521839    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:32.622530 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:32.622499    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:33.074672 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:33.074643    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:33.124496 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:33.124443    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-b8z2h" podStartSLOduration=10.593779732 podStartE2EDuration="41.124403858s" totalImagesPullingTime="30.530624126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:36:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:32.460364672 +0000 UTC m=+40.708365191" watchObservedRunningTime="2026-09-04 23:37:33.124403858 +0000 UTC m=+41.372404374"
Sep 04 23:37:33.124774 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:33.124752    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-krlhk"]
Sep 04 23:37:33.124960 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:33.124949    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:33.125085 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:33.125057    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:33.126285 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:33.125919    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fk66m"]
Sep 04 23:37:33.126285 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:33.126024    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:33.126285 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:33.126120    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:33.126504 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:33.126438    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mrmc2"]
Sep 04 23:37:33.126559 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:33.126522    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:33.126640 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:33.126621    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:33.142748 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:33.142685    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:33.267070 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:33.267036    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:33.318591 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:33.318562    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:33.462273 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:33.462205    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:33.562020 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:33.561838    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:34.402871 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:34.402840    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:34.498276 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:34.498250    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:34.663754 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:34.663686    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:34.763286 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:34.763258    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:35.168641 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:35.168608    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:35.168641 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:35.168639    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:35.168819 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:35.168723    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:35.168819 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:35.168721    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:35.168819 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:35.168786    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:35.169002 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:35.168933    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:35.977167 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:35.977133    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:36.028017 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:36.027995    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:36.813008 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:36.812975    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:36.873573 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:36.873547    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:37.168426 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.168396    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:37.168801 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.168500    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:37.168801 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:37.168517    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mrmc2" podUID="0b8a5cc2-85e1-4921-a572-71fa06de5611"
Sep 04 23:37:37.168801 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:37.168599    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fk66m" podUID="202d6558-e746-4911-9b99-f1b079343126"
Sep 04 23:37:37.168801 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.168653    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:37.168801 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:37.168712    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krlhk" podUID="8678835d-111a-4d74-84e3-372e11e75a50"
Sep 04 23:37:37.502541 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.502494    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-127.ec2.internal" event="NodeReady"
Sep 04 23:37:37.502640 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.502584    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 04 23:37:37.546272 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.546249    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dblvw"]
Sep 04 23:37:37.555858 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.555831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:37.568243 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.568225    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-np8fh"]
Sep 04 23:37:37.568353 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.568338    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.570780 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.570759    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 04 23:37:37.570878 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.570855    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 04 23:37:37.570964 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.570888    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4rhln\""
Sep 04 23:37:37.594319 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.594302    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-np8fh"
Sep 04 23:37:37.594417 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.594251    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dblvw"]
Sep 04 23:37:37.594478 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.594440    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-np8fh"]
Sep 04 23:37:37.597211 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.597195    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 04 23:37:37.597309 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.597230    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 04 23:37:37.597309 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.597248    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ttwxb\""
Sep 04 23:37:37.597576 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.597559    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 04 23:37:37.616493 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.616470    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:37.671087 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.671065    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:37.726749 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.726730    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4f200b7-9fb1-457f-9e3a-f85901981765-cert\") pod \"ingress-canary-np8fh\" (UID: \"f4f200b7-9fb1-457f-9e3a-f85901981765\") " pod="openshift-ingress-canary/ingress-canary-np8fh"
Sep 04 23:37:37.726843 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.726754    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2qzk\" (UniqueName: \"kubernetes.io/projected/f4f200b7-9fb1-457f-9e3a-f85901981765-kube-api-access-g2qzk\") pod \"ingress-canary-np8fh\" (UID: \"f4f200b7-9fb1-457f-9e3a-f85901981765\") " pod="openshift-ingress-canary/ingress-canary-np8fh"
Sep 04 23:37:37.726843 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.726771    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8hkn\" (UniqueName: \"kubernetes.io/projected/d08e3b44-f805-4369-becc-e73602a975c7-kube-api-access-d8hkn\") pod \"dns-default-dblvw\" (UID: \"d08e3b44-f805-4369-becc-e73602a975c7\") " pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.726843 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.726809    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d08e3b44-f805-4369-becc-e73602a975c7-metrics-tls\") pod \"dns-default-dblvw\" (UID: \"d08e3b44-f805-4369-becc-e73602a975c7\") " pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.726843 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.726841    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d08e3b44-f805-4369-becc-e73602a975c7-config-volume\") pod \"dns-default-dblvw\" (UID: \"d08e3b44-f805-4369-becc-e73602a975c7\") " pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.727022 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.726857    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08e3b44-f805-4369-becc-e73602a975c7-tmp-dir\") pod \"dns-default-dblvw\" (UID: \"d08e3b44-f805-4369-becc-e73602a975c7\") " pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.827132 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.827086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4f200b7-9fb1-457f-9e3a-f85901981765-cert\") pod \"ingress-canary-np8fh\" (UID: \"f4f200b7-9fb1-457f-9e3a-f85901981765\") " pod="openshift-ingress-canary/ingress-canary-np8fh"
Sep 04 23:37:37.827132 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.827115    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2qzk\" (UniqueName: \"kubernetes.io/projected/f4f200b7-9fb1-457f-9e3a-f85901981765-kube-api-access-g2qzk\") pod \"ingress-canary-np8fh\" (UID: \"f4f200b7-9fb1-457f-9e3a-f85901981765\") " pod="openshift-ingress-canary/ingress-canary-np8fh"
Sep 04 23:37:37.827254 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.827136    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8hkn\" (UniqueName: \"kubernetes.io/projected/d08e3b44-f805-4369-becc-e73602a975c7-kube-api-access-d8hkn\") pod \"dns-default-dblvw\" (UID: \"d08e3b44-f805-4369-becc-e73602a975c7\") " pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.827254 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.827184    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d08e3b44-f805-4369-becc-e73602a975c7-metrics-tls\") pod \"dns-default-dblvw\" (UID: \"d08e3b44-f805-4369-becc-e73602a975c7\") " pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.827254 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.827221    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d08e3b44-f805-4369-becc-e73602a975c7-config-volume\") pod \"dns-default-dblvw\" (UID: \"d08e3b44-f805-4369-becc-e73602a975c7\") " pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.827254 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.827244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08e3b44-f805-4369-becc-e73602a975c7-tmp-dir\") pod \"dns-default-dblvw\" (UID: \"d08e3b44-f805-4369-becc-e73602a975c7\") " pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.827541 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.827524    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08e3b44-f805-4369-becc-e73602a975c7-tmp-dir\") pod \"dns-default-dblvw\" (UID: \"d08e3b44-f805-4369-becc-e73602a975c7\") " pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.827790 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.827768    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d08e3b44-f805-4369-becc-e73602a975c7-config-volume\") pod \"dns-default-dblvw\" (UID: \"d08e3b44-f805-4369-becc-e73602a975c7\") " pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.831092 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.831072    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d08e3b44-f805-4369-becc-e73602a975c7-metrics-tls\") pod \"dns-default-dblvw\" (UID: \"d08e3b44-f805-4369-becc-e73602a975c7\") " pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.831179 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.831123    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4f200b7-9fb1-457f-9e3a-f85901981765-cert\") pod \"ingress-canary-np8fh\" (UID: \"f4f200b7-9fb1-457f-9e3a-f85901981765\") " pod="openshift-ingress-canary/ingress-canary-np8fh"
Sep 04 23:37:37.835018 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.834997    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2qzk\" (UniqueName: \"kubernetes.io/projected/f4f200b7-9fb1-457f-9e3a-f85901981765-kube-api-access-g2qzk\") pod \"ingress-canary-np8fh\" (UID: \"f4f200b7-9fb1-457f-9e3a-f85901981765\") " pod="openshift-ingress-canary/ingress-canary-np8fh"
Sep 04 23:37:37.835116 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.835089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8hkn\" (UniqueName: \"kubernetes.io/projected/d08e3b44-f805-4369-becc-e73602a975c7-kube-api-access-d8hkn\") pod \"dns-default-dblvw\" (UID: \"d08e3b44-f805-4369-becc-e73602a975c7\") " pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.876960 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.876936    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:37.902063 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:37.902046    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-np8fh"
Sep 04 23:37:38.200580 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:38.200558    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dblvw"]
Sep 04 23:37:38.203678 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:37:38.203654    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08e3b44_f805_4369_becc_e73602a975c7.slice/crio-2792599fc91421e74a4d223b41fd663c0a847664a7f9d095faa63ace4b2be783 WatchSource:0}: Error finding container 2792599fc91421e74a4d223b41fd663c0a847664a7f9d095faa63ace4b2be783: Status 404 returned error can't find the container with id 2792599fc91421e74a4d223b41fd663c0a847664a7f9d095faa63ace4b2be783
Sep 04 23:37:38.216296 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:38.216273    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-np8fh"]
Sep 04 23:37:38.219499 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:37:38.219480    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f200b7_9fb1_457f_9e3a_f85901981765.slice/crio-805cc23e256cabe81ac2b4e2e312f72f3f7e31447116ad8e7b75641634b13b4c WatchSource:0}: Error finding container 805cc23e256cabe81ac2b4e2e312f72f3f7e31447116ad8e7b75641634b13b4c: Status 404 returned error can't find the container with id 805cc23e256cabe81ac2b4e2e312f72f3f7e31447116ad8e7b75641634b13b4c
Sep 04 23:37:38.445107 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:38.445070    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-np8fh" event={"ID":"f4f200b7-9fb1-457f-9e3a-f85901981765","Type":"ContainerStarted","Data":"805cc23e256cabe81ac2b4e2e312f72f3f7e31447116ad8e7b75641634b13b4c"}
Sep 04 23:37:38.446059 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:38.446039    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dblvw" event={"ID":"d08e3b44-f805-4369-becc-e73602a975c7","Type":"ContainerStarted","Data":"2792599fc91421e74a4d223b41fd663c0a847664a7f9d095faa63ace4b2be783"}
Sep 04 23:37:39.168617 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:39.168580    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:39.168617 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:39.168615    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:39.168852 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:39.168740    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:39.173133 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:39.172788    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8jjtb\""
Sep 04 23:37:39.174586 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:39.173958    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 04 23:37:39.174586 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:39.174014    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 04 23:37:39.174586 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:39.174062    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 04 23:37:39.174586 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:39.174048    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 04 23:37:39.174586 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:39.174330    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vhhzf\""
Sep 04 23:37:40.472590 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.472552    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d46b6b549-4q6lt"]
Sep 04 23:37:40.486466 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.486431    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.487337 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.486489    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d46b6b549-4q6lt"]
Sep 04 23:37:40.489297 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.489275    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 04 23:37:40.490572 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.490234    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 04 23:37:40.490572 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.490278    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 04 23:37:40.490572 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.490297    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 04 23:37:40.490572 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.490300    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 04 23:37:40.490572 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.490358    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 04 23:37:40.490572 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.490559    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 04 23:37:40.490869 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.490560    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-68msp\""
Sep 04 23:37:40.498187 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.498162    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:40.556046 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.556000    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:40.648363 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.648336    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-console-config\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.648528 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.648381    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-oauth-serving-cert\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.648528 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.648405    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ea88a41-231e-48f5-998a-97b35c78670f-console-serving-cert\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.648528 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.648431    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-service-ca\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.648677 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.648522    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq2xh\" (UniqueName: \"kubernetes.io/projected/8ea88a41-231e-48f5-998a-97b35c78670f-kube-api-access-qq2xh\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.648677 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.648566    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ea88a41-231e-48f5-998a-97b35c78670f-console-oauth-config\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.749136 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.749051    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-console-config\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.749136 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.749103    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-oauth-serving-cert\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.749136 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.749134    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ea88a41-231e-48f5-998a-97b35c78670f-console-serving-cert\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.749393 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.749162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-service-ca\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.749393 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.749196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq2xh\" (UniqueName: \"kubernetes.io/projected/8ea88a41-231e-48f5-998a-97b35c78670f-kube-api-access-qq2xh\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.749393 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.749223    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ea88a41-231e-48f5-998a-97b35c78670f-console-oauth-config\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.749946 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.749892    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-console-config\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.750429 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.750410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-oauth-serving-cert\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.751008 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.750984    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-service-ca\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.752408 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.752387    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ea88a41-231e-48f5-998a-97b35c78670f-console-oauth-config\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.752936 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.752887    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ea88a41-231e-48f5-998a-97b35c78670f-console-serving-cert\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.761447 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.761428    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq2xh\" (UniqueName: \"kubernetes.io/projected/8ea88a41-231e-48f5-998a-97b35c78670f-kube-api-access-qq2xh\") pod \"console-d46b6b549-4q6lt\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") " pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:40.799311 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:40.799274    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:41.455301 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:41.455079    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-np8fh" event={"ID":"f4f200b7-9fb1-457f-9e3a-f85901981765","Type":"ContainerStarted","Data":"2a71e0bc60973dd8947ec2cb198097ce873b0916368c256a2df4a76b41569eec"}
Sep 04 23:37:41.457387 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:41.456668    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dblvw" event={"ID":"d08e3b44-f805-4369-becc-e73602a975c7","Type":"ContainerStarted","Data":"bcb8b48735ec8d83e8350ea984f1169b0ddbb5de01bdf2eb2e1bb9fa2d5cb760"}
Sep 04 23:37:41.469607 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:41.469559    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-np8fh" podStartSLOduration=1.389643413 podStartE2EDuration="4.469539814s" totalImagesPullingTime="3.079896401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:37:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:41.468997692 +0000 UTC m=+49.716998244" watchObservedRunningTime="2026-09-04 23:37:41.469539814 +0000 UTC m=+49.717540333"
Sep 04 23:37:41.558109 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:41.558083    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d46b6b549-4q6lt"]
Sep 04 23:37:41.562514 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:37:41.562485    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ea88a41_231e_48f5_998a_97b35c78670f.slice/crio-c081cd67a58b3a72d5a9012eac4e968fefbf2c0d3777f861b1e4917a642c2f10 WatchSource:0}: Error finding container c081cd67a58b3a72d5a9012eac4e968fefbf2c0d3777f861b1e4917a642c2f10: Status 404 returned error can't find the container with id c081cd67a58b3a72d5a9012eac4e968fefbf2c0d3777f861b1e4917a642c2f10
Sep 04 23:37:42.461636 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:42.461429    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dblvw" event={"ID":"d08e3b44-f805-4369-becc-e73602a975c7","Type":"ContainerStarted","Data":"6e95567b78e862bcaf6f16193fe6295c11db11be511535dc20885078d19a3308"}
Sep 04 23:37:42.461812 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:42.461655    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:42.462804 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:42.462775    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d46b6b549-4q6lt" event={"ID":"8ea88a41-231e-48f5-998a-97b35c78670f","Type":"ContainerStarted","Data":"c081cd67a58b3a72d5a9012eac4e968fefbf2c0d3777f861b1e4917a642c2f10"}
Sep 04 23:37:42.481313 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:42.481270    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dblvw" podStartSLOduration=2.392465185 podStartE2EDuration="5.481250886s" totalImagesPullingTime="3.088785701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:37:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:42.479595441 +0000 UTC m=+50.727595981" watchObservedRunningTime="2026-09-04 23:37:42.481250886 +0000 UTC m=+50.729251401"
Sep 04 23:37:45.460881 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.460850    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f9c6b6c86-8knnw"]
Sep 04 23:37:45.462660 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.462644    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.474071 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.474040    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 04 23:37:45.475499 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.475471    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f9c6b6c86-8knnw"]
Sep 04 23:37:45.476343 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.476318    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:45.477244 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.477224    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d46b6b549-4q6lt" event={"ID":"8ea88a41-231e-48f5-998a-97b35c78670f","Type":"ContainerStarted","Data":"e5aa82600d33eab693874364fc806ecdf1e4eb02ceec1123bd91953eea7fee58"}
Sep 04 23:37:45.506057 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.506015    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d46b6b549-4q6lt" podStartSLOduration=2.175985382 podStartE2EDuration="5.506001861s" totalImagesPullingTime="3.330016479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:45.50567879 +0000 UTC m=+53.753679325" watchObservedRunningTime="2026-09-04 23:37:45.506001861 +0000 UTC m=+53.754002376"
Sep 04 23:37:45.575092 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.575056    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:45.576914 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.576874    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-oauth-serving-cert\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.577003 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.576984    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx84s\" (UniqueName: \"kubernetes.io/projected/699f7ccc-de2d-40c0-9cae-599874b5c67e-kube-api-access-mx84s\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.577061 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.577037    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-serving-cert\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.577108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.577076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-service-ca\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.577108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.577096    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-config\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.577193 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.577124    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-oauth-config\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.577193 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.577166    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-trusted-ca-bundle\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.678215 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.678190    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-serving-cert\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.678296 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.678218    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-service-ca\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.678296 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.678236    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-config\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.678296 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.678253    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-oauth-config\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.678296 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.678272    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-trusted-ca-bundle\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.678480 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.678333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-oauth-serving-cert\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.678480 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.678405    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx84s\" (UniqueName: \"kubernetes.io/projected/699f7ccc-de2d-40c0-9cae-599874b5c67e-kube-api-access-mx84s\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.679029 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.679008    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-service-ca\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.679130 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.679026    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-oauth-serving-cert\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.679130 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.679064    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-config\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.679809 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.679788    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-trusted-ca-bundle\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.682263 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.682236    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-serving-cert\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.682361 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.682275    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-oauth-config\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.685530 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.685509    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx84s\" (UniqueName: \"kubernetes.io/projected/699f7ccc-de2d-40c0-9cae-599874b5c67e-kube-api-access-mx84s\") pod \"console-6f9c6b6c86-8knnw\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") " pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:45.776990 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:45.776937    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:46.045290 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:46.045071    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f9c6b6c86-8knnw"]
Sep 04 23:37:46.048614 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:37:46.048588    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod699f7ccc_de2d_40c0_9cae_599874b5c67e.slice/crio-f9cf491396ab2df53d84c67af5fbde707df469f2b45426c5b46193012f1f0ef6 WatchSource:0}: Error finding container f9cf491396ab2df53d84c67af5fbde707df469f2b45426c5b46193012f1f0ef6: Status 404 returned error can't find the container with id f9cf491396ab2df53d84c67af5fbde707df469f2b45426c5b46193012f1f0ef6
Sep 04 23:37:46.463741 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:46.463706    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:46.480976 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:46.480944    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9c6b6c86-8knnw" event={"ID":"699f7ccc-de2d-40c0-9cae-599874b5c67e","Type":"ContainerStarted","Data":"f11a9457906a74fa8b3d361da841d3f0bb91b8b7f1e2e265a7ce3d1094c7859e"}
Sep 04 23:37:46.481088 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:46.480993    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9c6b6c86-8knnw" event={"ID":"699f7ccc-de2d-40c0-9cae-599874b5c67e","Type":"ContainerStarted","Data":"f9cf491396ab2df53d84c67af5fbde707df469f2b45426c5b46193012f1f0ef6"}
Sep 04 23:37:46.497325 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:46.497286    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f9c6b6c86-8knnw" podStartSLOduration=1.497272425 podStartE2EDuration="1.497272425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:37:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:46.496989003 +0000 UTC m=+54.744989539" watchObservedRunningTime="2026-09-04 23:37:46.497272425 +0000 UTC m=+54.745272940"
Sep 04 23:37:46.564414 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:46.564383    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:47.004482 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:47.004449    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:47.055219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:47.055185    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:50.660199 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:50.660161    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:50.760973 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:50.760933    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:50.799713 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:50.799687    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:50.799802 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:50.799746    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:37:50.801251 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:50.801221    2564 patch_prober.go:28] interesting pod/console-d46b6b549-4q6lt container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.8:8443/health\": dial tcp 10.133.0.8:8443: connect: connection refused" start-of-body=
Sep 04 23:37:50.801361 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:50.801285    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-d46b6b549-4q6lt" podUID="8ea88a41-231e-48f5-998a-97b35c78670f" containerName="console" probeResult="failure" output="Get \"https://10.133.0.8:8443/health\": dial tcp 10.133.0.8:8443: connect: connection refused"
Sep 04 23:37:52.403546 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:52.403515    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lrgw5"
Sep 04 23:37:52.467585 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:52.467561    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dblvw"
Sep 04 23:37:52.548606 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:52.548575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:52.598814 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:52.598790    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:52.757783 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:52.757717    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:52.858419 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:52.858396    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:52.910276 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:52.910244    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:53.474158 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:53.474121    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:53.574603 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:53.574575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:53.642082 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:53.642059    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45"]
Sep 04 23:37:53.644605 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:53.644587    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45"
Sep 04 23:37:53.647147 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:53.647131    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-jvsmz\""
Sep 04 23:37:53.647248 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:53.647166    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 04 23:37:53.649616 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:53.649581    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:53.653528 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:53.653508    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45"]
Sep 04 23:37:53.731494 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:53.731444    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/92ec9bce-ddb1-4d7e-91c8-c963bbaebb41-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-qmp45\" (UID: \"92ec9bce-ddb1-4d7e-91c8-c963bbaebb41\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45"
Sep 04 23:37:53.831694 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:53.831665    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/92ec9bce-ddb1-4d7e-91c8-c963bbaebb41-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-qmp45\" (UID: \"92ec9bce-ddb1-4d7e-91c8-c963bbaebb41\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45"
Sep 04 23:37:53.831820 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:53.831806    2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 04 23:37:53.831875 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:53.831865    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ec9bce-ddb1-4d7e-91c8-c963bbaebb41-tls-certificates podName:92ec9bce-ddb1-4d7e-91c8-c963bbaebb41 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:54.331850138 +0000 UTC m=+62.579850638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/92ec9bce-ddb1-4d7e-91c8-c963bbaebb41-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-qmp45" (UID: "92ec9bce-ddb1-4d7e-91c8-c963bbaebb41") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 04 23:37:54.192804 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.192766    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:54.247697 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.247669    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:54.335679 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.335652    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/92ec9bce-ddb1-4d7e-91c8-c963bbaebb41-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-qmp45\" (UID: \"92ec9bce-ddb1-4d7e-91c8-c963bbaebb41\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45"
Sep 04 23:37:54.335834 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:54.335816    2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 04 23:37:54.335930 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:54.335893    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ec9bce-ddb1-4d7e-91c8-c963bbaebb41-tls-certificates podName:92ec9bce-ddb1-4d7e-91c8-c963bbaebb41 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:55.335872475 +0000 UTC m=+63.583872983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/92ec9bce-ddb1-4d7e-91c8-c963bbaebb41-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-qmp45" (UID: "92ec9bce-ddb1-4d7e-91c8-c963bbaebb41") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 04 23:37:54.706996 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.706965    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:54.875119 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.875083    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x7wjj"]
Sep 04 23:37:54.877230 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.877202    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:54.879740 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.879709    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 04 23:37:54.879834 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.879770    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qcf5d\""
Sep 04 23:37:54.881108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.881084    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 04 23:37:54.881233 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.881092    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 04 23:37:54.881233 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.881138    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 04 23:37:54.887329 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.887300    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x7wjj"]
Sep 04 23:37:54.893828 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.893803    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:54.939273 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.939249    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27p84\" (UniqueName: \"kubernetes.io/projected/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-api-access-27p84\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:54.939372 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.939284    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/222b3a9b-c948-4aad-88e1-4dde66f6f993-crio-socket\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:54.939372 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.939321    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:54.939463 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.939418    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/222b3a9b-c948-4aad-88e1-4dde66f6f993-data-volume\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:54.939506 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.939474    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:54.946295 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:54.946267    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:55.040208 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.040140    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:55.040305 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.040205    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27p84\" (UniqueName: \"kubernetes.io/projected/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-api-access-27p84\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:55.040305 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.040235    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/222b3a9b-c948-4aad-88e1-4dde66f6f993-crio-socket\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:55.040305 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:55.040260    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 04 23:37:55.040305 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.040276    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:55.040495 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.040326    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/222b3a9b-c948-4aad-88e1-4dde66f6f993-data-volume\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:55.040495 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:55.040343    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm podName:222b3a9b-c948-4aad-88e1-4dde66f6f993 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:55.540323772 +0000 UTC m=+63.788324284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x7wjj" (UID: "222b3a9b-c948-4aad-88e1-4dde66f6f993") : configmap "kube-rbac-proxy" not found
Sep 04 23:37:55.040495 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:55.040457    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 23:37:55.040681 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.040488    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/222b3a9b-c948-4aad-88e1-4dde66f6f993-crio-socket\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:55.040681 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:55.040506    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls podName:222b3a9b-c948-4aad-88e1-4dde66f6f993 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:55.54049116 +0000 UTC m=+63.788491670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7wjj" (UID: "222b3a9b-c948-4aad-88e1-4dde66f6f993") : secret "insights-runtime-extractor-tls" not found
Sep 04 23:37:55.040681 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.040617    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/222b3a9b-c948-4aad-88e1-4dde66f6f993-data-volume\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:55.050408 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.050389    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27p84\" (UniqueName: \"kubernetes.io/projected/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-api-access-27p84\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:55.342050 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.341989    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/92ec9bce-ddb1-4d7e-91c8-c963bbaebb41-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-qmp45\" (UID: \"92ec9bce-ddb1-4d7e-91c8-c963bbaebb41\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45"
Sep 04 23:37:55.344537 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.344506    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/92ec9bce-ddb1-4d7e-91c8-c963bbaebb41-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-qmp45\" (UID: \"92ec9bce-ddb1-4d7e-91c8-c963bbaebb41\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45"
Sep 04 23:37:55.454092 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.454063    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45"
Sep 04 23:37:55.539409 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.539378    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:55.544171 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.544149    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:55.544266 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.544212    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:55.544309 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:55.544290    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 04 23:37:55.544365 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:55.544355    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm podName:222b3a9b-c948-4aad-88e1-4dde66f6f993 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:56.544337656 +0000 UTC m=+64.792338153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x7wjj" (UID: "222b3a9b-c948-4aad-88e1-4dde66f6f993") : configmap "kube-rbac-proxy" not found
Sep 04 23:37:55.544419 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:55.544294    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 23:37:55.544458 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:55.544436    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls podName:222b3a9b-c948-4aad-88e1-4dde66f6f993 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:56.544423306 +0000 UTC m=+64.792423800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7wjj" (UID: "222b3a9b-c948-4aad-88e1-4dde66f6f993") : secret "insights-runtime-extractor-tls" not found
Sep 04 23:37:55.638573 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.638546    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:55.724738 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.724713    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45"]
Sep 04 23:37:55.728208 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:37:55.728187    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92ec9bce_ddb1_4d7e_91c8_c963bbaebb41.slice/crio-09914880ff31f8944e3e9982945d1476c2ce66d180fedf0f18a9bc2acdffba4a WatchSource:0}: Error finding container 09914880ff31f8944e3e9982945d1476c2ce66d180fedf0f18a9bc2acdffba4a: Status 404 returned error can't find the container with id 09914880ff31f8944e3e9982945d1476c2ce66d180fedf0f18a9bc2acdffba4a
Sep 04 23:37:55.777830 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.777800    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:55.777944 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.777876    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:37:55.779068 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.779048    2564 patch_prober.go:28] interesting pod/console-6f9c6b6c86-8knnw container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.9:8443/health\": dial tcp 10.133.0.9:8443: connect: connection refused" start-of-body=
Sep 04 23:37:55.779149 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:55.779088    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6f9c6b6c86-8knnw" podUID="699f7ccc-de2d-40c0-9cae-599874b5c67e" containerName="console" probeResult="failure" output="Get \"https://10.133.0.9:8443/health\": dial tcp 10.133.0.9:8443: connect: connection refused"
Sep 04 23:37:56.509468 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:56.509438    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45" event={"ID":"92ec9bce-ddb1-4d7e-91c8-c963bbaebb41","Type":"ContainerStarted","Data":"09914880ff31f8944e3e9982945d1476c2ce66d180fedf0f18a9bc2acdffba4a"}
Sep 04 23:37:56.551768 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:56.551734    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:56.551931 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:56.551814    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:56.551931 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:56.551873    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 04 23:37:56.552026 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:56.551939    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 23:37:56.552026 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:56.551963    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm podName:222b3a9b-c948-4aad-88e1-4dde66f6f993 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:58.551942683 +0000 UTC m=+66.799943194 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x7wjj" (UID: "222b3a9b-c948-4aad-88e1-4dde66f6f993") : configmap "kube-rbac-proxy" not found
Sep 04 23:37:56.552026 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:56.551982    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls podName:222b3a9b-c948-4aad-88e1-4dde66f6f993 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:58.551971341 +0000 UTC m=+66.799971855 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7wjj" (UID: "222b3a9b-c948-4aad-88e1-4dde66f6f993") : secret "insights-runtime-extractor-tls" not found
Sep 04 23:37:57.238784 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.238750    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f9c6b6c86-8knnw_699f7ccc-de2d-40c0-9cae-599874b5c67e/console/0.log"
Sep 04 23:37:57.440758 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.440730    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:57.440964 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.440920    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d46b6b549-4q6lt_8ea88a41-231e-48f5-998a-97b35c78670f/console/0.log"
Sep 04 23:37:57.513931 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.513855    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45" event={"ID":"92ec9bce-ddb1-4d7e-91c8-c963bbaebb41","Type":"ContainerStarted","Data":"748aef0f73fa83a416ab49e6d14fcc26607e5959345e0ee59e565c6e482028ea"}
Sep 04 23:37:57.514048 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.513993    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45"
Sep 04 23:37:57.518781 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.518762    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45"
Sep 04 23:37:57.532708 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.532661    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qmp45" podStartSLOduration=3.488234304 podStartE2EDuration="4.53265018s" totalImagesPullingTime="1.044415876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:37:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:37:57.531973432 +0000 UTC m=+65.779973972" watchObservedRunningTime="2026-09-04 23:37:57.53265018 +0000 UTC m=+65.780650716"
Sep 04 23:37:57.536951 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.536927    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:57.669953 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.669894    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:57.708440 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.708413    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"]
Sep 04 23:37:57.710272 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.710257    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:57.712863 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.712840    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 04 23:37:57.712999 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.712948    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 04 23:37:57.712999 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.712964    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2p5bn\""
Sep 04 23:37:57.712999 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.712980    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 04 23:37:57.713154 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.713018    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 04 23:37:57.713154 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.713087    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 04 23:37:57.719714 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.719692    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"]
Sep 04 23:37:57.740609 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.740584    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:57.760825 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.760805    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bb52f2e-0f57-400d-b35b-88264721d4f6-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-t4t9r\" (UID: \"3bb52f2e-0f57-400d-b35b-88264721d4f6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:57.760917 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.760828    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qwfp\" (UniqueName: \"kubernetes.io/projected/3bb52f2e-0f57-400d-b35b-88264721d4f6-kube-api-access-9qwfp\") pod \"prometheus-operator-6f4946db56-t4t9r\" (UID: \"3bb52f2e-0f57-400d-b35b-88264721d4f6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:57.760917 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.760850    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3bb52f2e-0f57-400d-b35b-88264721d4f6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-t4t9r\" (UID: \"3bb52f2e-0f57-400d-b35b-88264721d4f6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:57.760917 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.760865    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3bb52f2e-0f57-400d-b35b-88264721d4f6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-t4t9r\" (UID: \"3bb52f2e-0f57-400d-b35b-88264721d4f6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:57.861182 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.861161    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3bb52f2e-0f57-400d-b35b-88264721d4f6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-t4t9r\" (UID: \"3bb52f2e-0f57-400d-b35b-88264721d4f6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:57.861265 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.861186    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3bb52f2e-0f57-400d-b35b-88264721d4f6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-t4t9r\" (UID: \"3bb52f2e-0f57-400d-b35b-88264721d4f6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:57.861265 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.861260    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bb52f2e-0f57-400d-b35b-88264721d4f6-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-t4t9r\" (UID: \"3bb52f2e-0f57-400d-b35b-88264721d4f6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:57.861345 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.861278    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qwfp\" (UniqueName: \"kubernetes.io/projected/3bb52f2e-0f57-400d-b35b-88264721d4f6-kube-api-access-9qwfp\") pod \"prometheus-operator-6f4946db56-t4t9r\" (UID: \"3bb52f2e-0f57-400d-b35b-88264721d4f6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:57.861345 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:57.861289    2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 04 23:37:57.861345 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:57.861340    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bb52f2e-0f57-400d-b35b-88264721d4f6-prometheus-operator-tls podName:3bb52f2e-0f57-400d-b35b-88264721d4f6 nodeName:}" failed. No retries permitted until 2026-09-04 23:37:58.361327035 +0000 UTC m=+66.609327547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3bb52f2e-0f57-400d-b35b-88264721d4f6-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-t4t9r" (UID: "3bb52f2e-0f57-400d-b35b-88264721d4f6") : secret "prometheus-operator-tls" not found
Sep 04 23:37:57.864210 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.862313    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bb52f2e-0f57-400d-b35b-88264721d4f6-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-t4t9r\" (UID: \"3bb52f2e-0f57-400d-b35b-88264721d4f6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:57.865851 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.865830    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3bb52f2e-0f57-400d-b35b-88264721d4f6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-t4t9r\" (UID: \"3bb52f2e-0f57-400d-b35b-88264721d4f6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:57.875127 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.875105    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qwfp\" (UniqueName: \"kubernetes.io/projected/3bb52f2e-0f57-400d-b35b-88264721d4f6-kube-api-access-9qwfp\") pod \"prometheus-operator-6f4946db56-t4t9r\" (UID: \"3bb52f2e-0f57-400d-b35b-88264721d4f6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:57.962381 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.962355    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:57.962498 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.962392    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs\") pod \"network-metrics-daemon-fk66m\" (UID: \"202d6558-e746-4911-9b99-f1b079343126\") " pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:57.964835 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.964816    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 04 23:37:57.964950 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.964819    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 04 23:37:57.975107 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.975089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/202d6558-e746-4911-9b99-f1b079343126-metrics-certs\") pod \"network-metrics-daemon-fk66m\" (UID: \"202d6558-e746-4911-9b99-f1b079343126\") " pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:57.975178 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:57.975138    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8678835d-111a-4d74-84e3-372e11e75a50-original-pull-secret\") pod \"global-pull-secret-syncer-krlhk\" (UID: \"8678835d-111a-4d74-84e3-372e11e75a50\") " pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:58.063402 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.063370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vqvz\" (UniqueName: \"kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz\") pod \"network-check-target-mrmc2\" (UID: \"0b8a5cc2-85e1-4921-a572-71fa06de5611\") " pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:58.065880 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.065862    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 04 23:37:58.066974 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.066949    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:58.077051 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.077029    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 04 23:37:58.086352 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.086334    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8jjtb\""
Sep 04 23:37:58.086547 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.086531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vqvz\" (UniqueName: \"kubernetes.io/projected/0b8a5cc2-85e1-4921-a572-71fa06de5611-kube-api-access-5vqvz\") pod \"network-check-target-mrmc2\" (UID: \"0b8a5cc2-85e1-4921-a572-71fa06de5611\") " pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:58.092417 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.092402    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krlhk"
Sep 04 23:37:58.094413 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.094398    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fk66m"
Sep 04 23:37:58.101053 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.101034    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vhhzf\""
Sep 04 23:37:58.109367 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.109346    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:37:58.121482 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.121438    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:58.236110 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.236088    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dblvw_d08e3b44-f805-4369-becc-e73602a975c7/dns/0.log"
Sep 04 23:37:58.236278 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.236259    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:58.364652 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.364576    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fk66m"]
Sep 04 23:37:58.365617 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.365594    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3bb52f2e-0f57-400d-b35b-88264721d4f6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-t4t9r\" (UID: \"3bb52f2e-0f57-400d-b35b-88264721d4f6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:58.367448 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.367422    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-krlhk"]
Sep 04 23:37:58.368392 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.368372    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3bb52f2e-0f57-400d-b35b-88264721d4f6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-t4t9r\" (UID: \"3bb52f2e-0f57-400d-b35b-88264721d4f6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:58.368628 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:37:58.368611    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod202d6558_e746_4911_9b99_f1b079343126.slice/crio-a66d96d64c5d3108ce293f5e16825db863cb6a08bf3d729871c076ee3220a4eb WatchSource:0}: Error finding container a66d96d64c5d3108ce293f5e16825db863cb6a08bf3d729871c076ee3220a4eb: Status 404 returned error can't find the container with id a66d96d64c5d3108ce293f5e16825db863cb6a08bf3d729871c076ee3220a4eb
Sep 04 23:37:58.369997 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:37:58.369973    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8678835d_111a_4d74_84e3_372e11e75a50.slice/crio-7b905b8775f01b3f570afdc08989e82e8c3b197f3f96097e8548304f30870750 WatchSource:0}: Error finding container 7b905b8775f01b3f570afdc08989e82e8c3b197f3f96097e8548304f30870750: Status 404 returned error can't find the container with id 7b905b8775f01b3f570afdc08989e82e8c3b197f3f96097e8548304f30870750
Sep 04 23:37:58.387948 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.387922    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mrmc2"]
Sep 04 23:37:58.390510 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:37:58.390487    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b8a5cc2_85e1_4921_a572_71fa06de5611.slice/crio-d6def34a06af5834b1590e92d59cb0ece95fcfc8e350ebb55b62542058e1da7b WatchSource:0}: Error finding container d6def34a06af5834b1590e92d59cb0ece95fcfc8e350ebb55b62542058e1da7b: Status 404 returned error can't find the container with id d6def34a06af5834b1590e92d59cb0ece95fcfc8e350ebb55b62542058e1da7b
Sep 04 23:37:58.436250 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.436222    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dblvw_d08e3b44-f805-4369-becc-e73602a975c7/kube-rbac-proxy/0.log"
Sep 04 23:37:58.518487 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.518444    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-krlhk" event={"ID":"8678835d-111a-4d74-84e3-372e11e75a50","Type":"ContainerStarted","Data":"7b905b8775f01b3f570afdc08989e82e8c3b197f3f96097e8548304f30870750"}
Sep 04 23:37:58.519602 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.519581    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fk66m" event={"ID":"202d6558-e746-4911-9b99-f1b079343126","Type":"ContainerStarted","Data":"a66d96d64c5d3108ce293f5e16825db863cb6a08bf3d729871c076ee3220a4eb"}
Sep 04 23:37:58.520702 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.520673    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mrmc2" event={"ID":"0b8a5cc2-85e1-4921-a572-71fa06de5611","Type":"ContainerStarted","Data":"d6def34a06af5834b1590e92d59cb0ece95fcfc8e350ebb55b62542058e1da7b"}
Sep 04 23:37:58.567295 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.567264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:58.567434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.567326    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:37:58.567434 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:58.567382    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 04 23:37:58.567434 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:58.567415    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 23:37:58.567651 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:58.567448    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm podName:222b3a9b-c948-4aad-88e1-4dde66f6f993 nodeName:}" failed. No retries permitted until 2026-09-04 23:38:02.567432694 +0000 UTC m=+70.815433188 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x7wjj" (UID: "222b3a9b-c948-4aad-88e1-4dde66f6f993") : configmap "kube-rbac-proxy" not found
Sep 04 23:37:58.567651 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:37:58.567467    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls podName:222b3a9b-c948-4aad-88e1-4dde66f6f993 nodeName:}" failed. No retries permitted until 2026-09-04 23:38:02.567456788 +0000 UTC m=+70.815457283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7wjj" (UID: "222b3a9b-c948-4aad-88e1-4dde66f6f993") : secret "insights-runtime-extractor-tls" not found
Sep 04 23:37:58.620097 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.620014    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"
Sep 04 23:37:58.887135 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:58.886579    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-t4t9r"]
Sep 04 23:37:58.889542 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:37:58.889511    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bb52f2e_0f57_400d_b35b_88264721d4f6.slice/crio-0aa8749e78da0e9c60fbea2cf68973e131dc6c9eb950dbec4ff35a758a7a8fc1 WatchSource:0}: Error finding container 0aa8749e78da0e9c60fbea2cf68973e131dc6c9eb950dbec4ff35a758a7a8fc1: Status 404 returned error can't find the container with id 0aa8749e78da0e9c60fbea2cf68973e131dc6c9eb950dbec4ff35a758a7a8fc1
Sep 04 23:37:59.037295 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:59.037132    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2q7dw_57fd64bb-51ba-4701-81bc-fbff66028fb2/dns-node-resolver/0.log"
Sep 04 23:37:59.394309 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:59.394214    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:59.445729 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:59.445588    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:59.527371 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:59.527307    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r" event={"ID":"3bb52f2e-0f57-400d-b35b-88264721d4f6","Type":"ContainerStarted","Data":"0aa8749e78da0e9c60fbea2cf68973e131dc6c9eb950dbec4ff35a758a7a8fc1"}
Sep 04 23:37:59.859950 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:59.859891    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:37:59.960514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:37:59.960408    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:00.037519 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:00.037486    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4l98m_2b94086d-38f4-4be0-857d-ae657478079d/node-ca/0.log"
Sep 04 23:38:00.060941 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:00.060866    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:00.116096 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:00.116066    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:00.534546 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:00.534507    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fk66m" event={"ID":"202d6558-e746-4911-9b99-f1b079343126","Type":"ContainerStarted","Data":"21cb4bf991d09d60feb58a8a1b73fa1913b9988bfa871fe057ef61750ecace18"}
Sep 04 23:38:00.535004 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:00.534563    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fk66m" event={"ID":"202d6558-e746-4911-9b99-f1b079343126","Type":"ContainerStarted","Data":"754c50560c9cb0520dd99a86ce84ee1a0f9853bdc9e7a6bd1392b86a1b62512a"}
Sep 04 23:38:00.636550 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:00.636518    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:00.737035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:00.737008    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:00.807282 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:00.807213    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:38:00.812982 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:00.812961    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:38:00.826946 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:00.826876    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fk66m" podStartSLOduration=66.66401239 podStartE2EDuration="1m7.826862125s" totalImagesPullingTime="1.162849735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:38:00.550088392 +0000 UTC m=+68.798088937" watchObservedRunningTime="2026-09-04 23:38:00.826862125 +0000 UTC m=+69.074862640"
Sep 04 23:38:00.995827 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:00.995792    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:01.236524 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:01.236484    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-np8fh_f4f200b7-9fb1-457f-9e3a-f85901981765/serve-healthcheck-canary/0.log"
Sep 04 23:38:01.398039 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:01.397999    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:01.450398 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:01.450365    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:01.540238 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:01.540158    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r" event={"ID":"3bb52f2e-0f57-400d-b35b-88264721d4f6","Type":"ContainerStarted","Data":"0e75fe83b4dc00497ad28875a6a9ee149fe8b3a6ab9b629fccab8baf7c02e1a7"}
Sep 04 23:38:01.669831 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:01.669784    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:02.602944 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:02.602882    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:38:02.603385 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:02.602976    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:38:02.603385 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:38:02.603015    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 04 23:38:02.603385 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:38:02.603090    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls podName:222b3a9b-c948-4aad-88e1-4dde66f6f993 nodeName:}" failed. No retries permitted until 2026-09-04 23:38:10.603065853 +0000 UTC m=+78.851066361 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7wjj" (UID: "222b3a9b-c948-4aad-88e1-4dde66f6f993") : secret "insights-runtime-extractor-tls" not found
Sep 04 23:38:02.603385 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:38:02.603111    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 04 23:38:02.603385 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:38:02.603167    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm podName:222b3a9b-c948-4aad-88e1-4dde66f6f993 nodeName:}" failed. No retries permitted until 2026-09-04 23:38:10.603147512 +0000 UTC m=+78.851148035 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x7wjj" (UID: "222b3a9b-c948-4aad-88e1-4dde66f6f993") : configmap "kube-rbac-proxy" not found
Sep 04 23:38:03.030413 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:03.030371    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:03.130461 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:03.130425    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:03.674605 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:03.674577    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:03.774324 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:03.774293    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:04.116231 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:04.116197    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:04.216147 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:04.216118    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:04.316067 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:04.316045    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:04.367165 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:04.367097    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:04.551505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:04.551468    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-krlhk" event={"ID":"8678835d-111a-4d74-84e3-372e11e75a50","Type":"ContainerStarted","Data":"efcfbfdbc0a213c5fc707b1384c249e89ad9418f16a454be2e1db6dbbbae76dc"}
Sep 04 23:38:04.553021 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:04.553002    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mrmc2" event={"ID":"0b8a5cc2-85e1-4921-a572-71fa06de5611","Type":"ContainerStarted","Data":"9c3ffbd12482e7312941f71a9da45e9b4e23e06912fb40da39d7d600c9cf2e1a"}
Sep 04 23:38:04.553126 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:04.553044    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:38:04.554580 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:04.554563    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r" event={"ID":"3bb52f2e-0f57-400d-b35b-88264721d4f6","Type":"ContainerStarted","Data":"1bb73bc756200dd08782e30e6cd821f095216579ef29c1315a19ddbd949fcdd0"}
Sep 04 23:38:04.567755 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:04.567678    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-krlhk" podStartSLOduration=66.321040559 podStartE2EDuration="1m11.567666926s" totalImagesPullingTime="5.246626367s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:36:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:38:04.56701676 +0000 UTC m=+72.815017278" watchObservedRunningTime="2026-09-04 23:38:04.567666926 +0000 UTC m=+72.815667442"
Sep 04 23:38:04.581507 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:04.581466    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-t4t9r" podStartSLOduration=5.776691208 podStartE2EDuration="7.581453686s" totalImagesPullingTime="1.804762478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:38:04.581267427 +0000 UTC m=+72.829267965" watchObservedRunningTime="2026-09-04 23:38:04.581453686 +0000 UTC m=+72.829454201"
Sep 04 23:38:04.595264 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:04.595223    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mrmc2" podStartSLOduration=67.383363478 podStartE2EDuration="1m12.595209203s" totalImagesPullingTime="5.211845725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:38:04.594685283 +0000 UTC m=+72.842685819" watchObservedRunningTime="2026-09-04 23:38:04.595209203 +0000 UTC m=+72.843209746"
Sep 04 23:38:05.781474 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:05.781445    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:38:05.785287 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:05.785262    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:38:05.807167 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:05.807135    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:05.848471 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:05.848447    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d46b6b549-4q6lt"]
Sep 04 23:38:05.974565 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:05.974539    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:06.062358 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.062284    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:06.084765 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.084730    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-p658s"]
Sep 04 23:38:06.086954 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.086935    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.089089 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.089070    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 04 23:38:06.089381 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.089363    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 04 23:38:06.089619 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.089603    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pq79h\""
Sep 04 23:38:06.089714 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.089650    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 04 23:38:06.128606 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.128582    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.128686 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.128608    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-accelerators-collector-config\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.128686 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.128628    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec473b55-9e99-4b68-ae23-b7eae54dc361-sys\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.128775 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.128682    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-wtmp\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.128775 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.128712    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qccnl\" (UniqueName: \"kubernetes.io/projected/ec473b55-9e99-4b68-ae23-b7eae54dc361-kube-api-access-qccnl\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.128775 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.128746    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-textfile\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.128775 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.128764    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-tls\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.128937 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.128786    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec473b55-9e99-4b68-ae23-b7eae54dc361-metrics-client-ca\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.128937 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.128813    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec473b55-9e99-4b68-ae23-b7eae54dc361-root\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.145956 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.145932    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:06.229549 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.229525    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec473b55-9e99-4b68-ae23-b7eae54dc361-metrics-client-ca\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.229623 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.229559    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec473b55-9e99-4b68-ae23-b7eae54dc361-root\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.229623 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.229597    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.229700 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.229642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec473b55-9e99-4b68-ae23-b7eae54dc361-root\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.229700 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.229678    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-accelerators-collector-config\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.229766 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.229703    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec473b55-9e99-4b68-ae23-b7eae54dc361-sys\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.229766 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.229740    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-wtmp\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.229858 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.229764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qccnl\" (UniqueName: \"kubernetes.io/projected/ec473b55-9e99-4b68-ae23-b7eae54dc361-kube-api-access-qccnl\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.229858 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.229801    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-textfile\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.229858 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.229802    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec473b55-9e99-4b68-ae23-b7eae54dc361-sys\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.229858 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.229831    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-tls\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.230065 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:38:06.229953    2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 04 23:38:06.230065 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.229970    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-wtmp\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.230065 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:38:06.230014    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-tls podName:ec473b55-9e99-4b68-ae23-b7eae54dc361 nodeName:}" failed. No retries permitted until 2026-09-04 23:38:06.729996747 +0000 UTC m=+74.977997246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-tls") pod "node-exporter-p658s" (UID: "ec473b55-9e99-4b68-ae23-b7eae54dc361") : secret "node-exporter-tls" not found
Sep 04 23:38:06.230217 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.230181    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-textfile\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.230217 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.230207    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec473b55-9e99-4b68-ae23-b7eae54dc361-metrics-client-ca\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.230280 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.230207    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-accelerators-collector-config\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.232029 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.232012    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.238500 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.238085    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qccnl\" (UniqueName: \"kubernetes.io/projected/ec473b55-9e99-4b68-ae23-b7eae54dc361-kube-api-access-qccnl\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.243456 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.243431    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:06.342916 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.340604    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:06.441461 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.441433    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:06.543996 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.543963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:06.640257 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.640230    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:06.733626 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.733597    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-tls\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.735873 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.735853    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec473b55-9e99-4b68-ae23-b7eae54dc361-node-exporter-tls\") pod \"node-exporter-p658s\" (UID: \"ec473b55-9e99-4b68-ae23-b7eae54dc361\") " pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:06.840064 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.840044    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:06.995917 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:06.995813    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p658s"
Sep 04 23:38:07.005619 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:38:07.005594    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec473b55_9e99_4b68_ae23_b7eae54dc361.slice/crio-56b7f52fb9aaf4ad94010f65e4f2f7b9403f5ca15c03ca763e15d37874b9dc35 WatchSource:0}: Error finding container 56b7f52fb9aaf4ad94010f65e4f2f7b9403f5ca15c03ca763e15d37874b9dc35: Status 404 returned error can't find the container with id 56b7f52fb9aaf4ad94010f65e4f2f7b9403f5ca15c03ca763e15d37874b9dc35
Sep 04 23:38:07.028996 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.028970    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:07.115295 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.115262    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:07.121757 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.121738    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 23:38:07.125097 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.125079    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.127721 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.127676    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 04 23:38:07.127940 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.127890    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 04 23:38:07.128257 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.128239    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ww86f\""
Sep 04 23:38:07.128355 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.128318    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 04 23:38:07.128421 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.128334    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 04 23:38:07.128421 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.128367    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 04 23:38:07.128421 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.128391    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 04 23:38:07.128570 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.128505    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 04 23:38:07.128570 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.128554    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 04 23:38:07.132743 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.132725    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 04 23:38:07.141779 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.141760    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 23:38:07.237380 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.237357    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d98184a6-55e8-4aaa-a060-5128eec037d9-config-out\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.237502 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.237408    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.237502 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.237428    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-config-volume\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.237502 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.237470    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.237502 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.237491    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.237651 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.237543    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c2j9\" (UniqueName: \"kubernetes.io/projected/d98184a6-55e8-4aaa-a060-5128eec037d9-kube-api-access-9c2j9\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.237651 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.237561    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d98184a6-55e8-4aaa-a060-5128eec037d9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.237651 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.237579    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d98184a6-55e8-4aaa-a060-5128eec037d9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.237651 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.237594    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.237651 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.237623    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.237820 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.237654    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d98184a6-55e8-4aaa-a060-5128eec037d9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.237820 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.237679    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-web-config\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.237820 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.237709    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d98184a6-55e8-4aaa-a060-5128eec037d9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.338368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.338311    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d98184a6-55e8-4aaa-a060-5128eec037d9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.338368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.338337    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.338368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.338359    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.338532 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.338381    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d98184a6-55e8-4aaa-a060-5128eec037d9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.338532 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:38:07.338480    2564 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 04 23:38:07.338532 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.338514    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-web-config\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.338641 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:38:07.338541    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-main-tls podName:d98184a6-55e8-4aaa-a060-5128eec037d9 nodeName:}" failed. No retries permitted until 2026-09-04 23:38:07.838522566 +0000 UTC m=+76.086523061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "d98184a6-55e8-4aaa-a060-5128eec037d9") : secret "alertmanager-main-tls" not found
Sep 04 23:38:07.338641 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.338585    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d98184a6-55e8-4aaa-a060-5128eec037d9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.338745 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.338641    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d98184a6-55e8-4aaa-a060-5128eec037d9-config-out\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.338745 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.338670    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.338745 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.338697    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-config-volume\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.338745 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.338726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.338971 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.338750    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.338971 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.338801    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c2j9\" (UniqueName: \"kubernetes.io/projected/d98184a6-55e8-4aaa-a060-5128eec037d9-kube-api-access-9c2j9\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.338971 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.338832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d98184a6-55e8-4aaa-a060-5128eec037d9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.339119 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.339011    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d98184a6-55e8-4aaa-a060-5128eec037d9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.339178 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.339164    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d98184a6-55e8-4aaa-a060-5128eec037d9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.339978 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.339954    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d98184a6-55e8-4aaa-a060-5128eec037d9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.341864 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.341837    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d98184a6-55e8-4aaa-a060-5128eec037d9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.342008 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.341988    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.342077 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.341988    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-web-config\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.342298 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.342266    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d98184a6-55e8-4aaa-a060-5128eec037d9-config-out\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.342298 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.342286    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.342610 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.342594    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.342775 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.342757    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.343160 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.343144    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-config-volume\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.347912 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.347878    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c2j9\" (UniqueName: \"kubernetes.io/projected/d98184a6-55e8-4aaa-a060-5128eec037d9-kube-api-access-9c2j9\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.566341 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.566308    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p658s" event={"ID":"ec473b55-9e99-4b68-ae23-b7eae54dc361","Type":"ContainerStarted","Data":"56b7f52fb9aaf4ad94010f65e4f2f7b9403f5ca15c03ca763e15d37874b9dc35"}
Sep 04 23:38:07.568918 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.568848    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:07.843152 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.843114    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:07.845885 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:07.845856    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d98184a6-55e8-4aaa-a060-5128eec037d9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d98184a6-55e8-4aaa-a060-5128eec037d9\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:08.036356 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:08.036327    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 04 23:38:08.412008 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:08.410471    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 23:38:08.412008 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:38:08.411695    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd98184a6_55e8_4aaa_a060_5128eec037d9.slice/crio-c5b57e421873879763967b76a223442119716ed47fb6752769a552a325d9599a WatchSource:0}: Error finding container c5b57e421873879763967b76a223442119716ed47fb6752769a552a325d9599a: Status 404 returned error can't find the container with id c5b57e421873879763967b76a223442119716ed47fb6752769a552a325d9599a
Sep 04 23:38:08.571880 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:08.571851    2564 generic.go:356] "Generic (PLEG): container finished" podID="ec473b55-9e99-4b68-ae23-b7eae54dc361" containerID="efed1ff1dbbf93a1359bc7a2a6aee7f486d54d4744b4ce7e900006fb12d4a301" exitCode=0
Sep 04 23:38:08.572020 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:08.571944    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p658s" event={"ID":"ec473b55-9e99-4b68-ae23-b7eae54dc361","Type":"ContainerDied","Data":"efed1ff1dbbf93a1359bc7a2a6aee7f486d54d4744b4ce7e900006fb12d4a301"}
Sep 04 23:38:08.573250 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:08.573226    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d98184a6-55e8-4aaa-a060-5128eec037d9","Type":"ContainerStarted","Data":"c5b57e421873879763967b76a223442119716ed47fb6752769a552a325d9599a"}
Sep 04 23:38:09.088549 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:09.088468    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:09.149751 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:09.149708    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:09.374791 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:09.374750    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:09.524719 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:09.524677    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:09.578686 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:09.578660    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p658s" event={"ID":"ec473b55-9e99-4b68-ae23-b7eae54dc361","Type":"ContainerStarted","Data":"cbbbca32e9354246591af0f4d6aecce0188e38887dba298e4c1b9f7dc04ede50"}
Sep 04 23:38:09.578856 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:09.578711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p658s" event={"ID":"ec473b55-9e99-4b68-ae23-b7eae54dc361","Type":"ContainerStarted","Data":"ed0969d10ad85d9c8156f20e35d70c98d8c1a9cb08290542f8d0db9ae72f8f45"}
Sep 04 23:38:09.580258 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:09.580230    2564 generic.go:356] "Generic (PLEG): container finished" podID="d98184a6-55e8-4aaa-a060-5128eec037d9" containerID="ba1d472f0b5afc328e4b734812f3778302c0a7e74877bb4abc9c3cf81ad83f54" exitCode=0
Sep 04 23:38:09.580350 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:09.580269    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d98184a6-55e8-4aaa-a060-5128eec037d9","Type":"ContainerDied","Data":"ba1d472f0b5afc328e4b734812f3778302c0a7e74877bb4abc9c3cf81ad83f54"}
Sep 04 23:38:09.601927 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:09.601873    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-p658s" podStartSLOduration=2.380272616 podStartE2EDuration="3.601859958s" totalImagesPullingTime="1.151393287s" totalInitContainerRuntime="70.194055ms" isStatefulPod=true podCreationTimestamp="2026-09-04 23:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:38:09.600449239 +0000 UTC m=+77.848449755" watchObservedRunningTime="2026-09-04 23:38:09.601859958 +0000 UTC m=+77.849860473"
Sep 04 23:38:10.494150 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.494107    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:10.551190 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.551157    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:10.663913 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.663862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:38:10.664073 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.663982    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:38:10.664598 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.664571    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/222b3a9b-c948-4aad-88e1-4dde66f6f993-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:38:10.666618 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.666596    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/222b3a9b-c948-4aad-88e1-4dde66f6f993-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7wjj\" (UID: \"222b3a9b-c948-4aad-88e1-4dde66f6f993\") " pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:38:10.786969 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.786889    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x7wjj"
Sep 04 23:38:10.832855 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.832829    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8f65b9876-xpg7t"]
Sep 04 23:38:10.835153 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.835130    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.835475 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.835443    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:10.846936 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.846891    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8f65b9876-xpg7t"]
Sep 04 23:38:10.865195 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.865164    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-config\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.865323 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.865219    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-service-ca\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.865323 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.865245    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-serving-cert\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.865435 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.865313    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-oauth-config\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.865435 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.865348    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-trusted-ca-bundle\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.865435 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.865374    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-oauth-serving-cert\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.865435 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.865396    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8f9p\" (UniqueName: \"kubernetes.io/projected/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-kube-api-access-w8f9p\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.925410 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.925380    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:10.967331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.965869    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-service-ca\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.967331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.965942    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-serving-cert\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.967331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.965992    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-oauth-config\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.967331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.966018    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-trusted-ca-bundle\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.967331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.966044    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-oauth-serving-cert\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.967331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.966069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8f9p\" (UniqueName: \"kubernetes.io/projected/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-kube-api-access-w8f9p\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.967331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.966129    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-config\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.967331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.966752    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-service-ca\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.967331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.966772    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-config\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.967331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.967260    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-oauth-serving-cert\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.967331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.967295    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-trusted-ca-bundle\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.969680 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.969623    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-serving-cert\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.970186 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.970165    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-oauth-config\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:10.975398 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:10.975378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8f9p\" (UniqueName: \"kubernetes.io/projected/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-kube-api-access-w8f9p\") pod \"console-8f65b9876-xpg7t\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") " pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:11.028026 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.027977    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:11.100342 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.100313    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:11.152438 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.152408    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:11.165434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.165368    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:11.197847 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.197820    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x7wjj"]
Sep 04 23:38:11.201271 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:38:11.201073    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod222b3a9b_c948_4aad_88e1_4dde66f6f993.slice/crio-680ebe50d208043d8708c379a58a44b9dba20eb8a04d6eebd4967d4f794a9ab3 WatchSource:0}: Error finding container 680ebe50d208043d8708c379a58a44b9dba20eb8a04d6eebd4967d4f794a9ab3: Status 404 returned error can't find the container with id 680ebe50d208043d8708c379a58a44b9dba20eb8a04d6eebd4967d4f794a9ab3
Sep 04 23:38:11.265677 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.265620    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"]
Sep 04 23:38:11.267982 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.267950    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.270991 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.270965    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 04 23:38:11.271407 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.271387    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 04 23:38:11.271540 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.271522    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 04 23:38:11.271607 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.271593    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 04 23:38:11.271669 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.271388    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-rrszk\""
Sep 04 23:38:11.271797 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.271709    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 04 23:38:11.277348 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.277328    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 04 23:38:11.286555 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.286528    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"]
Sep 04 23:38:11.368227 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.368204    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.368321 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.368233    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t2rr\" (UniqueName: \"kubernetes.io/projected/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-kube-api-access-4t2rr\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.368321 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.368253    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-secret-telemeter-client\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.368321 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.368275    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-metrics-client-ca\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.368488 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.368329    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-serving-certs-ca-bundle\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.368488 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.368380    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.368488 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.368427    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-telemeter-client-tls\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.368488 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.368451    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-federate-client-tls\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.437394 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.437374    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8f65b9876-xpg7t"]
Sep 04 23:38:11.440251 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:38:11.440231    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0300bbe5_5700_4fd0_97c7_2f4a060bc2ed.slice/crio-c4a60c7dec29bbb2c8459f5274395db77638d4e1257fdbed334ec55648ee5119 WatchSource:0}: Error finding container c4a60c7dec29bbb2c8459f5274395db77638d4e1257fdbed334ec55648ee5119: Status 404 returned error can't find the container with id c4a60c7dec29bbb2c8459f5274395db77638d4e1257fdbed334ec55648ee5119
Sep 04 23:38:11.469311 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.469288    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-metrics-client-ca\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.469394 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.469328    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-serving-certs-ca-bundle\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.469394 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.469350    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.469394 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.469376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-telemeter-client-tls\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.469394 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.469390    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-federate-client-tls\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.469589 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.469429    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.469589 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.469457    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t2rr\" (UniqueName: \"kubernetes.io/projected/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-kube-api-access-4t2rr\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.469589 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.469483    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-secret-telemeter-client\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.470056 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.470007    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-serving-certs-ca-bundle\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.470322 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.470295    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-metrics-client-ca\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.470628 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.470561    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.472639 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.472617    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.472708 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.472669    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-telemeter-client-tls\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.472862 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.472844    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-federate-client-tls\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.472953 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.472878    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-secret-telemeter-client\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.482555 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.482531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t2rr\" (UniqueName: \"kubernetes.io/projected/4d676e9d-9fb7-4cab-aba5-86f51c28ace4-kube-api-access-4t2rr\") pod \"telemeter-client-5984c4d6f6-4zg6f\" (UID: \"4d676e9d-9fb7-4cab-aba5-86f51c28ace4\") " pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.579183 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.579152    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"
Sep 04 23:38:11.591252 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.591222    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d98184a6-55e8-4aaa-a060-5128eec037d9","Type":"ContainerStarted","Data":"b785b8083bd10839a1ad50284b42153c2b3108efe587f51a3a6e6447d502bc8a"}
Sep 04 23:38:11.591357 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.591303    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d98184a6-55e8-4aaa-a060-5128eec037d9","Type":"ContainerStarted","Data":"2d1bcbefa3a466c11b18866380985ee99ae4eb49871f737e27d8cb86410e9b68"}
Sep 04 23:38:11.591357 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.591341    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d98184a6-55e8-4aaa-a060-5128eec037d9","Type":"ContainerStarted","Data":"16b85bdc8512a27678e2c3d179a0ebe719d4a8037f809e11081859a2017b943e"}
Sep 04 23:38:11.591461 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.591384    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d98184a6-55e8-4aaa-a060-5128eec037d9","Type":"ContainerStarted","Data":"e18a814d9fb63e2847ff406cc7d538171e105b122a3a131c210022b4c3998087"}
Sep 04 23:38:11.591461 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.591423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d98184a6-55e8-4aaa-a060-5128eec037d9","Type":"ContainerStarted","Data":"3816b78a5ca2e950edc029c3fee2b44b58ad1765b03a4e53443d9a71ab57587a"}
Sep 04 23:38:11.592951 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.592929    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f65b9876-xpg7t" event={"ID":"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed","Type":"ContainerStarted","Data":"e5699133b5819dc68eba2cce6612e6da7804f228e5f03e07ef170a0a880511bb"}
Sep 04 23:38:11.593075 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.592970    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f65b9876-xpg7t" event={"ID":"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed","Type":"ContainerStarted","Data":"c4a60c7dec29bbb2c8459f5274395db77638d4e1257fdbed334ec55648ee5119"}
Sep 04 23:38:11.594740 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.594715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x7wjj" event={"ID":"222b3a9b-c948-4aad-88e1-4dde66f6f993","Type":"ContainerStarted","Data":"0beb759d441dca97b750de1c9cb6d28abbc264c159d2e244e523a6c1b9ca953b"}
Sep 04 23:38:11.594819 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.594770    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x7wjj" event={"ID":"222b3a9b-c948-4aad-88e1-4dde66f6f993","Type":"ContainerStarted","Data":"680ebe50d208043d8708c379a58a44b9dba20eb8a04d6eebd4967d4f794a9ab3"}
Sep 04 23:38:11.614096 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.614050    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8f65b9876-xpg7t" podStartSLOduration=1.6140352089999999 podStartE2EDuration="1.614035209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:38:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:38:11.613225737 +0000 UTC m=+79.861226282" watchObservedRunningTime="2026-09-04 23:38:11.614035209 +0000 UTC m=+79.862035724"
Sep 04 23:38:11.854055 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:11.853875    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f"]
Sep 04 23:38:11.944533 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:38:11.944472    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d676e9d_9fb7_4cab_aba5_86f51c28ace4.slice/crio-e723e9b355a7393245374ab4fae2c0b169631feeee6dd7f3dbcf2da05412295e WatchSource:0}: Error finding container e723e9b355a7393245374ab4fae2c0b169631feeee6dd7f3dbcf2da05412295e: Status 404 returned error can't find the container with id e723e9b355a7393245374ab4fae2c0b169631feeee6dd7f3dbcf2da05412295e
Sep 04 23:38:12.113632 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.112056    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:12.268949 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.268847    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:12.273445 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.273421    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 23:38:12.277847 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.277330    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.280529 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.280509    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 04 23:38:12.280637 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.280516    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 04 23:38:12.280997 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.280975    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 04 23:38:12.281089 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.280999    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 04 23:38:12.281089 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.281071    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 04 23:38:12.281225 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.281212    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bnn62\""
Sep 04 23:38:12.282472 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.282160    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 04 23:38:12.282472 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.282167    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 04 23:38:12.282472 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.282245    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 04 23:38:12.282472 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.282329    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 04 23:38:12.282472 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.282330    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e9431jep8pshi\""
Sep 04 23:38:12.282724 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.282573    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 04 23:38:12.283772 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.282892    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 04 23:38:12.283772 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.282926    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 04 23:38:12.283772 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.283260    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 04 23:38:12.288319 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.287312    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 04 23:38:12.289579 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.289556    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 04 23:38:12.296937 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.295956    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 23:38:12.329262 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.329237    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:12.375698 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.375674    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.375808 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.375709    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.375808 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.375725    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.375808 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.375739    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.375956 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.375844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-config\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.375956 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.375873    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9511166b-ed6c-40fa-916e-38dcf55e91e2-config-out\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.375956 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.375910    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.375956 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.375942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.376109 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.375959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.376109 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.375977    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.376109 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.375998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9511166b-ed6c-40fa-916e-38dcf55e91e2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.376109 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.376012    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.376109 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.376030    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5whlf\" (UniqueName: \"kubernetes.io/projected/9511166b-ed6c-40fa-916e-38dcf55e91e2-kube-api-access-5whlf\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.376109 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.376069    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-web-config\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.376109 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.376086    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.376344 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.376123    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.376344 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.376149    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.376344 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.376193    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.376344 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.376218    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.376344 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.376234    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.476685 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.476652    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9511166b-ed6c-40fa-916e-38dcf55e91e2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.476685 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.476683    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.476880 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.476704    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5whlf\" (UniqueName: \"kubernetes.io/projected/9511166b-ed6c-40fa-916e-38dcf55e91e2-kube-api-access-5whlf\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.476880 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.476722    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-web-config\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.476880 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.476744    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.476880 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.476768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.476880 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.476795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.477166 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.477145    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.477224 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.477208    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.477279 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.477245    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.477333 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.477287    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.477333 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.477318    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.477452 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.477342    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.477452 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.477364    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.477452 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.477415    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-config\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.477452 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.477446    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9511166b-ed6c-40fa-916e-38dcf55e91e2-config-out\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.478499 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.478139    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.478499 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.478221    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.478999 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.478977    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.479509 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.479484    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.479664 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.479625    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.480308 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.480284    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.481140 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.481115    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.481222 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.481182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.482536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.482513    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9511166b-ed6c-40fa-916e-38dcf55e91e2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.482536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.482733    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9511166b-ed6c-40fa-916e-38dcf55e91e2-config-out\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.482536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.482890    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-web-config\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.482536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.483009    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.482536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.483270    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.482536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.483430    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.482536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.483698    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.482536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.483858    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.482536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.484117    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.482536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.484308    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.482536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.484616    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9511166b-ed6c-40fa-916e-38dcf55e91e2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.485540 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.485516    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.485843 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.485826    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-config\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.486244 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.486225    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5whlf\" (UniqueName: \"kubernetes.io/projected/9511166b-ed6c-40fa-916e-38dcf55e91e2-kube-api-access-5whlf\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.487250 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.487228    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.487312 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.487254    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9511166b-ed6c-40fa-916e-38dcf55e91e2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9511166b-ed6c-40fa-916e-38dcf55e91e2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.591474 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.591380    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:12.603663 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.603623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d98184a6-55e8-4aaa-a060-5128eec037d9","Type":"ContainerStarted","Data":"bce975017747eb9c97dfe6097bcfb0bbb926bf5a2e09b6bfc4473cd4435a4c58"}
Sep 04 23:38:12.605478 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.605450    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f" event={"ID":"4d676e9d-9fb7-4cab-aba5-86f51c28ace4","Type":"ContainerStarted","Data":"e723e9b355a7393245374ab4fae2c0b169631feeee6dd7f3dbcf2da05412295e"}
Sep 04 23:38:12.607971 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.607944    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x7wjj" event={"ID":"222b3a9b-c948-4aad-88e1-4dde66f6f993","Type":"ContainerStarted","Data":"382ce2f0936eb25cd3742c795c5c1a1c1a9ee5c03137c182b75aa5288afd6f2c"}
Sep 04 23:38:12.629580 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.629415    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.375367292 podStartE2EDuration="5.629397225s" totalImagesPullingTime="3.242128165s" totalInitContainerRuntime="11.901768ms" isStatefulPod=false podCreationTimestamp="2026-09-04 23:38:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:38:12.627282774 +0000 UTC m=+80.875283309" watchObservedRunningTime="2026-09-04 23:38:12.629397225 +0000 UTC m=+80.877397767"
Sep 04 23:38:12.878976 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:12.878937    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 04 23:38:12.880787 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:38:12.880756    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9511166b_ed6c_40fa_916e_38dcf55e91e2.slice/crio-fd6b71ec1f7f939c22d00d18c963282fd240467a7db40963795df58c4989e8b5 WatchSource:0}: Error finding container fd6b71ec1f7f939c22d00d18c963282fd240467a7db40963795df58c4989e8b5: Status 404 returned error can't find the container with id fd6b71ec1f7f939c22d00d18c963282fd240467a7db40963795df58c4989e8b5
Sep 04 23:38:13.614435 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:13.614303    2564 generic.go:356] "Generic (PLEG): container finished" podID="9511166b-ed6c-40fa-916e-38dcf55e91e2" containerID="6e03b05f9a90d0b4d249a33d769df67ed59fd188f48d32bc695889b5f636aca1" exitCode=0
Sep 04 23:38:13.614435 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:13.614385    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9511166b-ed6c-40fa-916e-38dcf55e91e2","Type":"ContainerDied","Data":"6e03b05f9a90d0b4d249a33d769df67ed59fd188f48d32bc695889b5f636aca1"}
Sep 04 23:38:13.614922 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:13.614460    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9511166b-ed6c-40fa-916e-38dcf55e91e2","Type":"ContainerStarted","Data":"fd6b71ec1f7f939c22d00d18c963282fd240467a7db40963795df58c4989e8b5"}
Sep 04 23:38:14.078323 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.078222    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:14.132996 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.132958    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:14.622243 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.622161    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f" event={"ID":"4d676e9d-9fb7-4cab-aba5-86f51c28ace4","Type":"ContainerStarted","Data":"24286492e8fb555c18f3b994945c7afc83d6821685316d5432ad32529b81287d"}
Sep 04 23:38:14.622243 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.622222    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f" event={"ID":"4d676e9d-9fb7-4cab-aba5-86f51c28ace4","Type":"ContainerStarted","Data":"5b450c7a4e7d807469baf4ada282d46b6a23b3d708f14ea2924df030659d249f"}
Sep 04 23:38:14.622768 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.622253    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f" event={"ID":"4d676e9d-9fb7-4cab-aba5-86f51c28ace4","Type":"ContainerStarted","Data":"0718e83001f9b6273b009a5ea8b578222911237ecfdb6869a0e3b4a945c2ec92"}
Sep 04 23:38:14.624716 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.624684    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x7wjj" event={"ID":"222b3a9b-c948-4aad-88e1-4dde66f6f993","Type":"ContainerStarted","Data":"02527ab308356ff656fc89f495a703dc09b642153e6c012dc0345ad4bd82e28b"}
Sep 04 23:38:14.643411 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.643343    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5984c4d6f6-4zg6f" podStartSLOduration=1.340685767 podStartE2EDuration="3.64332862s" totalImagesPullingTime="2.302642853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:38:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:38:14.641973654 +0000 UTC m=+82.889974192" watchObservedRunningTime="2026-09-04 23:38:14.64332862 +0000 UTC m=+82.891329137"
Sep 04 23:38:14.658688 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.658648    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-x7wjj" podStartSLOduration=17.748351871 podStartE2EDuration="20.658635512s" totalImagesPullingTime="2.910283641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 23:37:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:38:14.65781613 +0000 UTC m=+82.905816672" watchObservedRunningTime="2026-09-04 23:38:14.658635512 +0000 UTC m=+82.906636027"
Sep 04 23:38:14.738453 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.738433    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8f65b9876-xpg7t"]
Sep 04 23:38:14.761850 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.761822    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c8668cf5b-52fr4"]
Sep 04 23:38:14.765490 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.765475    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.774776 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.774756    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c8668cf5b-52fr4"]
Sep 04 23:38:14.801279 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.801255    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-service-ca\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.801366 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.801284    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-trusted-ca-bundle\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.801366 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.801323    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-serving-cert\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.801436 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.801387    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxdph\" (UniqueName: \"kubernetes.io/projected/f1a45b0f-be4e-4750-acb3-acb51ce6f272-kube-api-access-vxdph\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.801436 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.801424    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-config\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.801490 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.801455    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-oauth-serving-cert\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.801536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.801521    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-oauth-config\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.902322 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.902260    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-oauth-serving-cert\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.902506 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.902322    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-oauth-config\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.902506 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.902345    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-service-ca\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.902506 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.902369    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-trusted-ca-bundle\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.902675 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.902509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-serving-cert\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.902675 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.902553    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxdph\" (UniqueName: \"kubernetes.io/projected/f1a45b0f-be4e-4750-acb3-acb51ce6f272-kube-api-access-vxdph\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.902675 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.902586    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-config\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.903097 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.903069    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-oauth-serving-cert\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.903182 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.903147    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-service-ca\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.903683 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.903663    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-config\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.904027 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.903983    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-trusted-ca-bundle\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.905236 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.905214    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-oauth-config\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.905823 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.905803    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-serving-cert\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:14.911160 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:14.911137    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxdph\" (UniqueName: \"kubernetes.io/projected/f1a45b0f-be4e-4750-acb3-acb51ce6f272-kube-api-access-vxdph\") pod \"console-c8668cf5b-52fr4\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") " pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:15.075137 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:15.075105    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:15.141432 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:15.141391    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:15.239232 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:15.239157    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:15.365034 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:15.365002    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c8668cf5b-52fr4"]
Sep 04 23:38:15.367862 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:38:15.367833    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1a45b0f_be4e_4750_acb3_acb51ce6f272.slice/crio-a16aab52fd0c585c47e30975e8e11363f7aaee404b2125c24966afdc039eb5a8 WatchSource:0}: Error finding container a16aab52fd0c585c47e30975e8e11363f7aaee404b2125c24966afdc039eb5a8: Status 404 returned error can't find the container with id a16aab52fd0c585c47e30975e8e11363f7aaee404b2125c24966afdc039eb5a8
Sep 04 23:38:15.633152 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:15.633114    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8668cf5b-52fr4" event={"ID":"f1a45b0f-be4e-4750-acb3-acb51ce6f272","Type":"ContainerStarted","Data":"a4533f1280cb3fe28e2ea63f9997a58fa62bf508095d3a0e6d905d7c51eea6f8"}
Sep 04 23:38:15.633590 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:15.633175    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8668cf5b-52fr4" event={"ID":"f1a45b0f-be4e-4750-acb3-acb51ce6f272","Type":"ContainerStarted","Data":"a16aab52fd0c585c47e30975e8e11363f7aaee404b2125c24966afdc039eb5a8"}
Sep 04 23:38:15.649324 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:15.649281    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c8668cf5b-52fr4" podStartSLOduration=1.649264767 podStartE2EDuration="1.649264767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:38:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:38:15.648947638 +0000 UTC m=+83.896948166" watchObservedRunningTime="2026-09-04 23:38:15.649264767 +0000 UTC m=+83.897265283"
Sep 04 23:38:15.754771 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:15.754736    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:15.806065 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:15.806029    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:17.645174 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:17.645141    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9511166b-ed6c-40fa-916e-38dcf55e91e2","Type":"ContainerStarted","Data":"748451a725576fac5807c72353216aaadf7fe0e45266644bf2894fc39dfd33bd"}
Sep 04 23:38:18.652019 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:18.651988    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9511166b-ed6c-40fa-916e-38dcf55e91e2","Type":"ContainerStarted","Data":"256061f927156f46f85b50fb9e7167f4e2b3f9a0f64c9d2a489b5072046bdab5"}
Sep 04 23:38:20.664473 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:20.664429    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9511166b-ed6c-40fa-916e-38dcf55e91e2","Type":"ContainerStarted","Data":"d57ca7aeaf13d2b8650b174c1bd66b30d70d334f1d70ff5d58b37fb03eb78275"}
Sep 04 23:38:20.664855 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:20.664534    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9511166b-ed6c-40fa-916e-38dcf55e91e2","Type":"ContainerStarted","Data":"2518df643940bba038bb2131f294fe33a29f98694aef4483c6329b5eff1e14e8"}
Sep 04 23:38:20.664855 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:20.664567    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9511166b-ed6c-40fa-916e-38dcf55e91e2","Type":"ContainerStarted","Data":"cbad046c72fe6036a72fc86675b082bfb1dd7ff1ce73ac6da90f249ac78035c5"}
Sep 04 23:38:20.664855 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:20.664597    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9511166b-ed6c-40fa-916e-38dcf55e91e2","Type":"ContainerStarted","Data":"c9fb89867e872edc211c433da5f22557a6268ccbfaf220387552e83b46cd8fc0"}
Sep 04 23:38:20.692461 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:20.692402    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.7958415949999997 podStartE2EDuration="8.692380689s" totalImagesPullingTime="5.871285207s" totalInitContainerRuntime="25.253887ms" isStatefulPod=false podCreationTimestamp="2026-09-04 23:38:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:38:20.690113181 +0000 UTC m=+88.938113696" watchObservedRunningTime="2026-09-04 23:38:20.692380689 +0000 UTC m=+88.940381206"
Sep 04 23:38:21.152848 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:21.152818    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:22.591737 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:22.591705    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:38:25.075229 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:25.075182    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:25.075607 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:25.075261    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:25.080077 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:25.080053    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:25.487808 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:25.487776    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:25.538989 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:25.538961    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:25.687637 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:25.687616    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:38:25.730013 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:25.729982    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f9c6b6c86-8knnw"]
Sep 04 23:38:25.833938 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:25.833846    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:25.935178 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:25.935153    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:25.988027 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:25.987994    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:30.868069 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:30.868017    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d46b6b549-4q6lt" podUID="8ea88a41-231e-48f5-998a-97b35c78670f" containerName="console" containerID="cri-o://e5aa82600d33eab693874364fc806ecdf1e4eb02ceec1123bd91953eea7fee58" gracePeriod=15
Sep 04 23:38:31.116873 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.116846    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:38:31.242260 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.242186    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-console-config\") pod \"8ea88a41-231e-48f5-998a-97b35c78670f\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") "
Sep 04 23:38:31.242402 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.242257    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qq2xh\" (UniqueName: \"kubernetes.io/projected/8ea88a41-231e-48f5-998a-97b35c78670f-kube-api-access-qq2xh\") pod \"8ea88a41-231e-48f5-998a-97b35c78670f\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") "
Sep 04 23:38:31.242402 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.242294    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ea88a41-231e-48f5-998a-97b35c78670f-console-serving-cert\") pod \"8ea88a41-231e-48f5-998a-97b35c78670f\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") "
Sep 04 23:38:31.242402 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.242325    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-oauth-serving-cert\") pod \"8ea88a41-231e-48f5-998a-97b35c78670f\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") "
Sep 04 23:38:31.242402 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.242349    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ea88a41-231e-48f5-998a-97b35c78670f-console-oauth-config\") pod \"8ea88a41-231e-48f5-998a-97b35c78670f\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") "
Sep 04 23:38:31.242402 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.242387    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-service-ca\") pod \"8ea88a41-231e-48f5-998a-97b35c78670f\" (UID: \"8ea88a41-231e-48f5-998a-97b35c78670f\") "
Sep 04 23:38:31.242675 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.242650    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-console-config" (OuterVolumeSpecName: "console-config") pod "8ea88a41-231e-48f5-998a-97b35c78670f" (UID: "8ea88a41-231e-48f5-998a-97b35c78670f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:38:31.242786 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.242741    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8ea88a41-231e-48f5-998a-97b35c78670f" (UID: "8ea88a41-231e-48f5-998a-97b35c78670f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:38:31.243088 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.242873    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-service-ca" (OuterVolumeSpecName: "service-ca") pod "8ea88a41-231e-48f5-998a-97b35c78670f" (UID: "8ea88a41-231e-48f5-998a-97b35c78670f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:38:31.244733 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.244706    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ea88a41-231e-48f5-998a-97b35c78670f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8ea88a41-231e-48f5-998a-97b35c78670f" (UID: "8ea88a41-231e-48f5-998a-97b35c78670f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:38:31.244847 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.244785    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ea88a41-231e-48f5-998a-97b35c78670f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8ea88a41-231e-48f5-998a-97b35c78670f" (UID: "8ea88a41-231e-48f5-998a-97b35c78670f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:38:31.244847 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.244819    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ea88a41-231e-48f5-998a-97b35c78670f-kube-api-access-qq2xh" (OuterVolumeSpecName: "kube-api-access-qq2xh") pod "8ea88a41-231e-48f5-998a-97b35c78670f" (UID: "8ea88a41-231e-48f5-998a-97b35c78670f"). InnerVolumeSpecName "kube-api-access-qq2xh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:38:31.343325 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.343301    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-console-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:31.343325 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.343321    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq2xh\" (UniqueName: \"kubernetes.io/projected/8ea88a41-231e-48f5-998a-97b35c78670f-kube-api-access-qq2xh\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:31.343443 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.343332    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ea88a41-231e-48f5-998a-97b35c78670f-console-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:31.343443 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.343341    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-oauth-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:31.343443 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.343349    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ea88a41-231e-48f5-998a-97b35c78670f-console-oauth-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:31.343443 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.343357    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea88a41-231e-48f5-998a-97b35c78670f-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:31.711277 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.711248    2564 generic.go:356] "Generic (PLEG): container finished" podID="8ea88a41-231e-48f5-998a-97b35c78670f" containerID="e5aa82600d33eab693874364fc806ecdf1e4eb02ceec1123bd91953eea7fee58" exitCode=0
Sep 04 23:38:31.711406 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.711291    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d46b6b549-4q6lt"
Sep 04 23:38:31.711406 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.711334    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d46b6b549-4q6lt" event={"ID":"8ea88a41-231e-48f5-998a-97b35c78670f","Type":"ContainerDied","Data":"e5aa82600d33eab693874364fc806ecdf1e4eb02ceec1123bd91953eea7fee58"}
Sep 04 23:38:31.711406 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.711387    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d46b6b549-4q6lt" event={"ID":"8ea88a41-231e-48f5-998a-97b35c78670f","Type":"ContainerDied","Data":"c081cd67a58b3a72d5a9012eac4e968fefbf2c0d3777f861b1e4917a642c2f10"}
Sep 04 23:38:31.711535 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.711420    2564 scope.go:117] "RemoveContainer" containerID="e5aa82600d33eab693874364fc806ecdf1e4eb02ceec1123bd91953eea7fee58"
Sep 04 23:38:31.721144 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.721123    2564 scope.go:117] "RemoveContainer" containerID="e5aa82600d33eab693874364fc806ecdf1e4eb02ceec1123bd91953eea7fee58"
Sep 04 23:38:31.721472 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:38:31.721451    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5aa82600d33eab693874364fc806ecdf1e4eb02ceec1123bd91953eea7fee58\": container with ID starting with e5aa82600d33eab693874364fc806ecdf1e4eb02ceec1123bd91953eea7fee58 not found: ID does not exist" containerID="e5aa82600d33eab693874364fc806ecdf1e4eb02ceec1123bd91953eea7fee58"
Sep 04 23:38:31.721547 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.721485    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5aa82600d33eab693874364fc806ecdf1e4eb02ceec1123bd91953eea7fee58"} err="failed to get container status \"e5aa82600d33eab693874364fc806ecdf1e4eb02ceec1123bd91953eea7fee58\": rpc error: code = NotFound desc = could not find container \"e5aa82600d33eab693874364fc806ecdf1e4eb02ceec1123bd91953eea7fee58\": container with ID starting with e5aa82600d33eab693874364fc806ecdf1e4eb02ceec1123bd91953eea7fee58 not found: ID does not exist"
Sep 04 23:38:31.732004 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.731982    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d46b6b549-4q6lt"]
Sep 04 23:38:31.734964 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:31.734942    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d46b6b549-4q6lt"]
Sep 04 23:38:32.176479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:32.176448    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ea88a41-231e-48f5-998a-97b35c78670f" path="/var/lib/kubelet/pods/8ea88a41-231e-48f5-998a-97b35c78670f/volumes"
Sep 04 23:38:34.965650 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:34.965612    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:35.017313 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:35.017289    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:38:35.561561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:35.561532    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mrmc2"
Sep 04 23:38:39.758184 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:39.758147    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8f65b9876-xpg7t" podUID="0300bbe5-5700-4fd0-97c7-2f4a060bc2ed" containerName="console" containerID="cri-o://e5699133b5819dc68eba2cce6612e6da7804f228e5f03e07ef170a0a880511bb" gracePeriod=15
Sep 04 23:38:39.999054 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:39.999032    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:40.108231 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.108212    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-config\") pod \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") "
Sep 04 23:38:40.108397 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.108251    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8f9p\" (UniqueName: \"kubernetes.io/projected/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-kube-api-access-w8f9p\") pod \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") "
Sep 04 23:38:40.108397 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.108277    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-trusted-ca-bundle\") pod \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") "
Sep 04 23:38:40.108397 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.108296    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-service-ca\") pod \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") "
Sep 04 23:38:40.108552 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.108438    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-oauth-config\") pod \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") "
Sep 04 23:38:40.108552 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.108468    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-oauth-serving-cert\") pod \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") "
Sep 04 23:38:40.108552 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.108493    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-serving-cert\") pod \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\" (UID: \"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed\") "
Sep 04 23:38:40.108772 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.108745    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-config" (OuterVolumeSpecName: "console-config") pod "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed" (UID: "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:38:40.108827 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.108754    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed" (UID: "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:38:40.108827 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.108813    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed" (UID: "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:38:40.108978 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.108960    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-service-ca" (OuterVolumeSpecName: "service-ca") pod "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed" (UID: "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:38:40.110774 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.110754    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed" (UID: "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:38:40.111105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.111083    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-kube-api-access-w8f9p" (OuterVolumeSpecName: "kube-api-access-w8f9p") pod "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed" (UID: "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed"). InnerVolumeSpecName "kube-api-access-w8f9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:38:40.111200 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.111093    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed" (UID: "0300bbe5-5700-4fd0-97c7-2f4a060bc2ed"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:38:40.209482 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.209457    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:40.209482 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.209477    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8f9p\" (UniqueName: \"kubernetes.io/projected/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-kube-api-access-w8f9p\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:40.209609 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.209488    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-trusted-ca-bundle\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:40.209609 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.209498    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:40.209609 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.209507    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-oauth-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:40.209609 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.209516    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-oauth-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:40.209609 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.209523    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed-console-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:40.749450 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.749415    2564 generic.go:356] "Generic (PLEG): container finished" podID="0300bbe5-5700-4fd0-97c7-2f4a060bc2ed" containerID="e5699133b5819dc68eba2cce6612e6da7804f228e5f03e07ef170a0a880511bb" exitCode=0
Sep 04 23:38:40.749656 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.749478    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f65b9876-xpg7t"
Sep 04 23:38:40.749656 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.749507    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f65b9876-xpg7t" event={"ID":"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed","Type":"ContainerDied","Data":"e5699133b5819dc68eba2cce6612e6da7804f228e5f03e07ef170a0a880511bb"}
Sep 04 23:38:40.749656 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.749565    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f65b9876-xpg7t" event={"ID":"0300bbe5-5700-4fd0-97c7-2f4a060bc2ed","Type":"ContainerDied","Data":"c4a60c7dec29bbb2c8459f5274395db77638d4e1257fdbed334ec55648ee5119"}
Sep 04 23:38:40.749656 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.749586    2564 scope.go:117] "RemoveContainer" containerID="e5699133b5819dc68eba2cce6612e6da7804f228e5f03e07ef170a0a880511bb"
Sep 04 23:38:40.758140 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.758123    2564 scope.go:117] "RemoveContainer" containerID="e5699133b5819dc68eba2cce6612e6da7804f228e5f03e07ef170a0a880511bb"
Sep 04 23:38:40.758408 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:38:40.758386    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5699133b5819dc68eba2cce6612e6da7804f228e5f03e07ef170a0a880511bb\": container with ID starting with e5699133b5819dc68eba2cce6612e6da7804f228e5f03e07ef170a0a880511bb not found: ID does not exist" containerID="e5699133b5819dc68eba2cce6612e6da7804f228e5f03e07ef170a0a880511bb"
Sep 04 23:38:40.758646 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.758414    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5699133b5819dc68eba2cce6612e6da7804f228e5f03e07ef170a0a880511bb"} err="failed to get container status \"e5699133b5819dc68eba2cce6612e6da7804f228e5f03e07ef170a0a880511bb\": rpc error: code = NotFound desc = could not find container \"e5699133b5819dc68eba2cce6612e6da7804f228e5f03e07ef170a0a880511bb\": container with ID starting with e5699133b5819dc68eba2cce6612e6da7804f228e5f03e07ef170a0a880511bb not found: ID does not exist"
Sep 04 23:38:40.765424 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.765388    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8f65b9876-xpg7t"]
Sep 04 23:38:40.767361 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:40.767339    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8f65b9876-xpg7t"]
Sep 04 23:38:42.173156 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:42.173122    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0300bbe5-5700-4fd0-97c7-2f4a060bc2ed" path="/var/lib/kubelet/pods/0300bbe5-5700-4fd0-97c7-2f4a060bc2ed/volumes"
Sep 04 23:38:50.750170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:50.750109    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f9c6b6c86-8knnw" podUID="699f7ccc-de2d-40c0-9cae-599874b5c67e" containerName="console" containerID="cri-o://f11a9457906a74fa8b3d361da841d3f0bb91b8b7f1e2e265a7ce3d1094c7859e" gracePeriod=15
Sep 04 23:38:51.005716 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.005661    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:38:51.092825 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.092796    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-config\") pod \"699f7ccc-de2d-40c0-9cae-599874b5c67e\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") "
Sep 04 23:38:51.092981 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.092837    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-oauth-serving-cert\") pod \"699f7ccc-de2d-40c0-9cae-599874b5c67e\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") "
Sep 04 23:38:51.092981 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.092861    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mx84s\" (UniqueName: \"kubernetes.io/projected/699f7ccc-de2d-40c0-9cae-599874b5c67e-kube-api-access-mx84s\") pod \"699f7ccc-de2d-40c0-9cae-599874b5c67e\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") "
Sep 04 23:38:51.092981 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.092923    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-serving-cert\") pod \"699f7ccc-de2d-40c0-9cae-599874b5c67e\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") "
Sep 04 23:38:51.092981 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.092959    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-oauth-config\") pod \"699f7ccc-de2d-40c0-9cae-599874b5c67e\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") "
Sep 04 23:38:51.093198 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.093026    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-trusted-ca-bundle\") pod \"699f7ccc-de2d-40c0-9cae-599874b5c67e\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") "
Sep 04 23:38:51.093198 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.093052    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-service-ca\") pod \"699f7ccc-de2d-40c0-9cae-599874b5c67e\" (UID: \"699f7ccc-de2d-40c0-9cae-599874b5c67e\") "
Sep 04 23:38:51.093297 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.093194    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-config" (OuterVolumeSpecName: "console-config") pod "699f7ccc-de2d-40c0-9cae-599874b5c67e" (UID: "699f7ccc-de2d-40c0-9cae-599874b5c67e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:38:51.093297 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.093240    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "699f7ccc-de2d-40c0-9cae-599874b5c67e" (UID: "699f7ccc-de2d-40c0-9cae-599874b5c67e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:38:51.093475 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.093430    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "699f7ccc-de2d-40c0-9cae-599874b5c67e" (UID: "699f7ccc-de2d-40c0-9cae-599874b5c67e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:38:51.093751 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.093476    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-service-ca" (OuterVolumeSpecName: "service-ca") pod "699f7ccc-de2d-40c0-9cae-599874b5c67e" (UID: "699f7ccc-de2d-40c0-9cae-599874b5c67e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:38:51.093751 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.093579    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:51.093751 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.093597    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:51.093751 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.093606    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-oauth-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:51.093751 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.093614    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/699f7ccc-de2d-40c0-9cae-599874b5c67e-trusted-ca-bundle\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:51.095229 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.095209    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "699f7ccc-de2d-40c0-9cae-599874b5c67e" (UID: "699f7ccc-de2d-40c0-9cae-599874b5c67e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:38:51.095310 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.095255    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/699f7ccc-de2d-40c0-9cae-599874b5c67e-kube-api-access-mx84s" (OuterVolumeSpecName: "kube-api-access-mx84s") pod "699f7ccc-de2d-40c0-9cae-599874b5c67e" (UID: "699f7ccc-de2d-40c0-9cae-599874b5c67e"). InnerVolumeSpecName "kube-api-access-mx84s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:38:51.095310 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.095266    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "699f7ccc-de2d-40c0-9cae-599874b5c67e" (UID: "699f7ccc-de2d-40c0-9cae-599874b5c67e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:38:51.194301 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.194277    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:51.194301 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.194297    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/699f7ccc-de2d-40c0-9cae-599874b5c67e-console-oauth-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:51.194426 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.194307    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mx84s\" (UniqueName: \"kubernetes.io/projected/699f7ccc-de2d-40c0-9cae-599874b5c67e-kube-api-access-mx84s\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:38:51.792331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.792298    2564 generic.go:356] "Generic (PLEG): container finished" podID="699f7ccc-de2d-40c0-9cae-599874b5c67e" containerID="f11a9457906a74fa8b3d361da841d3f0bb91b8b7f1e2e265a7ce3d1094c7859e" exitCode=0
Sep 04 23:38:51.792736 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.792350    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9c6b6c86-8knnw"
Sep 04 23:38:51.792736 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.792366    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9c6b6c86-8knnw" event={"ID":"699f7ccc-de2d-40c0-9cae-599874b5c67e","Type":"ContainerDied","Data":"f11a9457906a74fa8b3d361da841d3f0bb91b8b7f1e2e265a7ce3d1094c7859e"}
Sep 04 23:38:51.792736 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.792409    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9c6b6c86-8knnw" event={"ID":"699f7ccc-de2d-40c0-9cae-599874b5c67e","Type":"ContainerDied","Data":"f9cf491396ab2df53d84c67af5fbde707df469f2b45426c5b46193012f1f0ef6"}
Sep 04 23:38:51.792736 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.792435    2564 scope.go:117] "RemoveContainer" containerID="f11a9457906a74fa8b3d361da841d3f0bb91b8b7f1e2e265a7ce3d1094c7859e"
Sep 04 23:38:51.804889 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.804871    2564 scope.go:117] "RemoveContainer" containerID="f11a9457906a74fa8b3d361da841d3f0bb91b8b7f1e2e265a7ce3d1094c7859e"
Sep 04 23:38:51.805231 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:38:51.805210    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f11a9457906a74fa8b3d361da841d3f0bb91b8b7f1e2e265a7ce3d1094c7859e\": container with ID starting with f11a9457906a74fa8b3d361da841d3f0bb91b8b7f1e2e265a7ce3d1094c7859e not found: ID does not exist" containerID="f11a9457906a74fa8b3d361da841d3f0bb91b8b7f1e2e265a7ce3d1094c7859e"
Sep 04 23:38:51.805313 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.805240    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f11a9457906a74fa8b3d361da841d3f0bb91b8b7f1e2e265a7ce3d1094c7859e"} err="failed to get container status \"f11a9457906a74fa8b3d361da841d3f0bb91b8b7f1e2e265a7ce3d1094c7859e\": rpc error: code = NotFound desc = could not find container \"f11a9457906a74fa8b3d361da841d3f0bb91b8b7f1e2e265a7ce3d1094c7859e\": container with ID starting with f11a9457906a74fa8b3d361da841d3f0bb91b8b7f1e2e265a7ce3d1094c7859e not found: ID does not exist"
Sep 04 23:38:51.813807 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.813785    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f9c6b6c86-8knnw"]
Sep 04 23:38:51.816238 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:51.816217    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f9c6b6c86-8knnw"]
Sep 04 23:38:52.173190 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:52.173161    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="699f7ccc-de2d-40c0-9cae-599874b5c67e" path="/var/lib/kubelet/pods/699f7ccc-de2d-40c0-9cae-599874b5c67e/volumes"
Sep 04 23:38:59.973665 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:38:59.973628    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:00.023534 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:00.023500    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:12.592056 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:12.592017    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:39:12.611466 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:12.611440    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:39:12.893029 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:12.893003    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 04 23:39:22.151502 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:22.151451    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:22.203084 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:22.203050    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:25.120677 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:25.120645    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:25.221191 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:25.221154    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:30.562483 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.562447    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68f776d88d-s226f"]
Sep 04 23:39:30.562840 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.562712    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0300bbe5-5700-4fd0-97c7-2f4a060bc2ed" containerName="console"
Sep 04 23:39:30.562840 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.562722    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0300bbe5-5700-4fd0-97c7-2f4a060bc2ed" containerName="console"
Sep 04 23:39:30.562840 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.562769    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ea88a41-231e-48f5-998a-97b35c78670f" containerName="console"
Sep 04 23:39:30.562840 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.562775    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ea88a41-231e-48f5-998a-97b35c78670f" containerName="console"
Sep 04 23:39:30.562840 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.562789    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="699f7ccc-de2d-40c0-9cae-599874b5c67e" containerName="console"
Sep 04 23:39:30.562840 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.562795    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="699f7ccc-de2d-40c0-9cae-599874b5c67e" containerName="console"
Sep 04 23:39:30.562840 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.562839    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="699f7ccc-de2d-40c0-9cae-599874b5c67e" containerName="console"
Sep 04 23:39:30.563075 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.562847    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ea88a41-231e-48f5-998a-97b35c78670f" containerName="console"
Sep 04 23:39:30.563075 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.562855    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0300bbe5-5700-4fd0-97c7-2f4a060bc2ed" containerName="console"
Sep 04 23:39:30.565472 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.565445    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.576814 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.576786    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68f776d88d-s226f"]
Sep 04 23:39:30.620361 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.620334    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:30.678273 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.678248    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01b597dd-b90d-46d5-903c-d6760312794d-console-oauth-config\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.678379 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.678289    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-trusted-ca-bundle\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.678379 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.678306    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nplrp\" (UniqueName: \"kubernetes.io/projected/01b597dd-b90d-46d5-903c-d6760312794d-kube-api-access-nplrp\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.678379 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.678354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-oauth-serving-cert\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.678542 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.678396    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-console-config\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.678542 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.678450    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-service-ca\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.678542 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.678489    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01b597dd-b90d-46d5-903c-d6760312794d-console-serving-cert\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.756330 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.756281    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:30.779155 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.779133    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-service-ca\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.779240 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.779162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01b597dd-b90d-46d5-903c-d6760312794d-console-serving-cert\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.779240 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.779189    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01b597dd-b90d-46d5-903c-d6760312794d-console-oauth-config\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.779240 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.779220    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-trusted-ca-bundle\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.779391 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.779241    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nplrp\" (UniqueName: \"kubernetes.io/projected/01b597dd-b90d-46d5-903c-d6760312794d-kube-api-access-nplrp\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.779391 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.779263    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-oauth-serving-cert\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.779391 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.779278    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-console-config\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.779957 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.779938    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-console-config\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.780053 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.780035    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-oauth-serving-cert\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.780271 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.780251    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-service-ca\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.780422 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.780401    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-trusted-ca-bundle\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.781869 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.781842    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01b597dd-b90d-46d5-903c-d6760312794d-console-serving-cert\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.781962 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.781940    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01b597dd-b90d-46d5-903c-d6760312794d-console-oauth-config\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.786879 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.786858    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nplrp\" (UniqueName: \"kubernetes.io/projected/01b597dd-b90d-46d5-903c-d6760312794d-kube-api-access-nplrp\") pod \"console-68f776d88d-s226f\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") " pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:30.804933 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.804892    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:30.874999 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:30.874978    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:31.142721 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:31.142641    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68f776d88d-s226f"]
Sep 04 23:39:31.147304 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:39:31.147275    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b597dd_b90d_46d5_903c_d6760312794d.slice/crio-c7a9dd3d79da6a95de27b5fd3a6683df5f35bef58c834a001486a17c6bc43136 WatchSource:0}: Error finding container c7a9dd3d79da6a95de27b5fd3a6683df5f35bef58c834a001486a17c6bc43136: Status 404 returned error can't find the container with id c7a9dd3d79da6a95de27b5fd3a6683df5f35bef58c834a001486a17c6bc43136
Sep 04 23:39:31.366794 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:31.366763    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:31.418665 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:31.418560    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:31.952005 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:31.951975    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f776d88d-s226f" event={"ID":"01b597dd-b90d-46d5-903c-d6760312794d","Type":"ContainerStarted","Data":"ec10b47a99033ef9b56ab6964319303ac39e0b3bc9266a736a1e2a973eaf481e"}
Sep 04 23:39:31.952371 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:31.952052    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f776d88d-s226f" event={"ID":"01b597dd-b90d-46d5-903c-d6760312794d","Type":"ContainerStarted","Data":"c7a9dd3d79da6a95de27b5fd3a6683df5f35bef58c834a001486a17c6bc43136"}
Sep 04 23:39:31.968203 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:31.968155    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68f776d88d-s226f" podStartSLOduration=1.968140269 podStartE2EDuration="1.968140269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:39:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:39:31.967067274 +0000 UTC m=+160.215067823" watchObservedRunningTime="2026-09-04 23:39:31.968140269 +0000 UTC m=+160.216140859"
Sep 04 23:39:34.052482 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:34.052444    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:34.101077 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:34.101044    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:40.495660 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:40.495622    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:40.595458 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:40.595428    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:40.875652 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:40.875624    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:40.875821 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:40.875699    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:40.880485 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:40.880458    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:40.989157 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:40.989127    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:39:41.034092 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:41.034050    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c8668cf5b-52fr4"]
Sep 04 23:39:41.153707 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:41.153610    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:41.201698 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:41.201673    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:41.260829 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:41.260790    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:39:42.246593 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:39:42.246533    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:40:06.054661 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.054541    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c8668cf5b-52fr4" podUID="f1a45b0f-be4e-4750-acb3-acb51ce6f272" containerName="console" containerID="cri-o://a4533f1280cb3fe28e2ea63f9997a58fa62bf508095d3a0e6d905d7c51eea6f8" gracePeriod=15
Sep 04 23:40:06.289274 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.289253    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:40:06.437008 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.436983    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-serving-cert\") pod \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") "
Sep 04 23:40:06.437167 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.437020    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-oauth-config\") pod \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") "
Sep 04 23:40:06.437167 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.437055    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-oauth-serving-cert\") pod \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") "
Sep 04 23:40:06.437167 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.437082    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-trusted-ca-bundle\") pod \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") "
Sep 04 23:40:06.437167 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.437111    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxdph\" (UniqueName: \"kubernetes.io/projected/f1a45b0f-be4e-4750-acb3-acb51ce6f272-kube-api-access-vxdph\") pod \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") "
Sep 04 23:40:06.437375 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.437181    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-config\") pod \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") "
Sep 04 23:40:06.437375 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.437206    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-service-ca\") pod \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\" (UID: \"f1a45b0f-be4e-4750-acb3-acb51ce6f272\") "
Sep 04 23:40:06.437538 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.437509    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f1a45b0f-be4e-4750-acb3-acb51ce6f272" (UID: "f1a45b0f-be4e-4750-acb3-acb51ce6f272"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:40:06.437600 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.437546    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f1a45b0f-be4e-4750-acb3-acb51ce6f272" (UID: "f1a45b0f-be4e-4750-acb3-acb51ce6f272"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:40:06.437672 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.437560    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-config" (OuterVolumeSpecName: "console-config") pod "f1a45b0f-be4e-4750-acb3-acb51ce6f272" (UID: "f1a45b0f-be4e-4750-acb3-acb51ce6f272"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:40:06.437940 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.437831    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-service-ca" (OuterVolumeSpecName: "service-ca") pod "f1a45b0f-be4e-4750-acb3-acb51ce6f272" (UID: "f1a45b0f-be4e-4750-acb3-acb51ce6f272"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:40:06.439327 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.439306    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f1a45b0f-be4e-4750-acb3-acb51ce6f272" (UID: "f1a45b0f-be4e-4750-acb3-acb51ce6f272"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:40:06.439868 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.439847    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f1a45b0f-be4e-4750-acb3-acb51ce6f272" (UID: "f1a45b0f-be4e-4750-acb3-acb51ce6f272"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:40:06.439960 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.439863    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1a45b0f-be4e-4750-acb3-acb51ce6f272-kube-api-access-vxdph" (OuterVolumeSpecName: "kube-api-access-vxdph") pod "f1a45b0f-be4e-4750-acb3-acb51ce6f272" (UID: "f1a45b0f-be4e-4750-acb3-acb51ce6f272"). InnerVolumeSpecName "kube-api-access-vxdph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:40:06.537877 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.537856    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:40:06.537877 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.537876    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:40:06.537877 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.537884    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:40:06.538065 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.537916    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f1a45b0f-be4e-4750-acb3-acb51ce6f272-console-oauth-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:40:06.538065 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.537929    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-oauth-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:40:06.538065 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.537937    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1a45b0f-be4e-4750-acb3-acb51ce6f272-trusted-ca-bundle\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:40:06.538065 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:06.537944    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxdph\" (UniqueName: \"kubernetes.io/projected/f1a45b0f-be4e-4750-acb3-acb51ce6f272-kube-api-access-vxdph\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:40:07.083729 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:07.083690    2564 generic.go:356] "Generic (PLEG): container finished" podID="f1a45b0f-be4e-4750-acb3-acb51ce6f272" containerID="a4533f1280cb3fe28e2ea63f9997a58fa62bf508095d3a0e6d905d7c51eea6f8" exitCode=0
Sep 04 23:40:07.084148 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:07.083736    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8668cf5b-52fr4"
Sep 04 23:40:07.084148 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:07.083778    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8668cf5b-52fr4" event={"ID":"f1a45b0f-be4e-4750-acb3-acb51ce6f272","Type":"ContainerDied","Data":"a4533f1280cb3fe28e2ea63f9997a58fa62bf508095d3a0e6d905d7c51eea6f8"}
Sep 04 23:40:07.084148 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:07.083834    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8668cf5b-52fr4" event={"ID":"f1a45b0f-be4e-4750-acb3-acb51ce6f272","Type":"ContainerDied","Data":"a16aab52fd0c585c47e30975e8e11363f7aaee404b2125c24966afdc039eb5a8"}
Sep 04 23:40:07.084148 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:07.083862    2564 scope.go:117] "RemoveContainer" containerID="a4533f1280cb3fe28e2ea63f9997a58fa62bf508095d3a0e6d905d7c51eea6f8"
Sep 04 23:40:07.092565 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:07.092548    2564 scope.go:117] "RemoveContainer" containerID="a4533f1280cb3fe28e2ea63f9997a58fa62bf508095d3a0e6d905d7c51eea6f8"
Sep 04 23:40:07.092848 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:40:07.092828    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4533f1280cb3fe28e2ea63f9997a58fa62bf508095d3a0e6d905d7c51eea6f8\": container with ID starting with a4533f1280cb3fe28e2ea63f9997a58fa62bf508095d3a0e6d905d7c51eea6f8 not found: ID does not exist" containerID="a4533f1280cb3fe28e2ea63f9997a58fa62bf508095d3a0e6d905d7c51eea6f8"
Sep 04 23:40:07.092886 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:07.092859    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4533f1280cb3fe28e2ea63f9997a58fa62bf508095d3a0e6d905d7c51eea6f8"} err="failed to get container status \"a4533f1280cb3fe28e2ea63f9997a58fa62bf508095d3a0e6d905d7c51eea6f8\": rpc error: code = NotFound desc = could not find container \"a4533f1280cb3fe28e2ea63f9997a58fa62bf508095d3a0e6d905d7c51eea6f8\": container with ID starting with a4533f1280cb3fe28e2ea63f9997a58fa62bf508095d3a0e6d905d7c51eea6f8 not found: ID does not exist"
Sep 04 23:40:07.104041 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:07.104017    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c8668cf5b-52fr4"]
Sep 04 23:40:07.107629 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:07.107607    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c8668cf5b-52fr4"]
Sep 04 23:40:08.173504 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:08.173467    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1a45b0f-be4e-4750-acb3-acb51ce6f272" path="/var/lib/kubelet/pods/f1a45b0f-be4e-4750-acb3-acb51ce6f272/volumes"
Sep 04 23:40:13.304351 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:13.304314    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:40:20.232224 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:20.232183    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:40:20.329083 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:20.329054    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:40:52.033450 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:52.033417    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:40:53.391812 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:53.391764    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:40:55.643041 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:55.643001    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:40:55.743042 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:55.743005    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:40:57.445305 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:57.445262    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:40:57.547251 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:57.547218    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:40:57.669767 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:57.669738    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:40:57.762116 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:40:57.762049    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:00.022728 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:00.022686    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:00.237121 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:00.237090    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:02.434678 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:02.434618    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:02.492384 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:02.492341    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:07.625890 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:07.625857    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:08.576668 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.576629    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rq9hz"]
Sep 04 23:41:08.576959 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.576944    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1a45b0f-be4e-4750-acb3-acb51ce6f272" containerName="console"
Sep 04 23:41:08.577014 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.576961    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1a45b0f-be4e-4750-acb3-acb51ce6f272" containerName="console"
Sep 04 23:41:08.577060 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.577019    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1a45b0f-be4e-4750-acb3-acb51ce6f272" containerName="console"
Sep 04 23:41:08.581146 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.581125    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rq9hz"
Sep 04 23:41:08.583683 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.583651    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 04 23:41:08.583683 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.583675    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5nms6\""
Sep 04 23:41:08.584732 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.584708    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 04 23:41:08.589636 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.589609    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rq9hz"]
Sep 04 23:41:08.658131 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.658103    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgc5h\" (UniqueName: \"kubernetes.io/projected/e34199b8-d8f7-4c47-92af-487e03848425-kube-api-access-lgc5h\") pod \"cert-manager-55b4d954d5-rq9hz\" (UID: \"e34199b8-d8f7-4c47-92af-487e03848425\") " pod="cert-manager/cert-manager-55b4d954d5-rq9hz"
Sep 04 23:41:08.658445 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.658155    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e34199b8-d8f7-4c47-92af-487e03848425-bound-sa-token\") pod \"cert-manager-55b4d954d5-rq9hz\" (UID: \"e34199b8-d8f7-4c47-92af-487e03848425\") " pod="cert-manager/cert-manager-55b4d954d5-rq9hz"
Sep 04 23:41:08.759518 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.759494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgc5h\" (UniqueName: \"kubernetes.io/projected/e34199b8-d8f7-4c47-92af-487e03848425-kube-api-access-lgc5h\") pod \"cert-manager-55b4d954d5-rq9hz\" (UID: \"e34199b8-d8f7-4c47-92af-487e03848425\") " pod="cert-manager/cert-manager-55b4d954d5-rq9hz"
Sep 04 23:41:08.759606 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.759539    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e34199b8-d8f7-4c47-92af-487e03848425-bound-sa-token\") pod \"cert-manager-55b4d954d5-rq9hz\" (UID: \"e34199b8-d8f7-4c47-92af-487e03848425\") " pod="cert-manager/cert-manager-55b4d954d5-rq9hz"
Sep 04 23:41:08.767165 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.767142    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgc5h\" (UniqueName: \"kubernetes.io/projected/e34199b8-d8f7-4c47-92af-487e03848425-kube-api-access-lgc5h\") pod \"cert-manager-55b4d954d5-rq9hz\" (UID: \"e34199b8-d8f7-4c47-92af-487e03848425\") " pod="cert-manager/cert-manager-55b4d954d5-rq9hz"
Sep 04 23:41:08.767609 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.767589    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e34199b8-d8f7-4c47-92af-487e03848425-bound-sa-token\") pod \"cert-manager-55b4d954d5-rq9hz\" (UID: \"e34199b8-d8f7-4c47-92af-487e03848425\") " pod="cert-manager/cert-manager-55b4d954d5-rq9hz"
Sep 04 23:41:08.901147 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:08.901126    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rq9hz"
Sep 04 23:41:09.167391 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:09.167368    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rq9hz"]
Sep 04 23:41:09.169734 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:41:09.169712    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode34199b8_d8f7_4c47_92af_487e03848425.slice/crio-60e40b7799eaa8743120d2bf4604cbc803503eba2bf733daf96164edaba1b26b WatchSource:0}: Error finding container 60e40b7799eaa8743120d2bf4604cbc803503eba2bf733daf96164edaba1b26b: Status 404 returned error can't find the container with id 60e40b7799eaa8743120d2bf4604cbc803503eba2bf733daf96164edaba1b26b
Sep 04 23:41:09.316885 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:09.316856    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rq9hz" event={"ID":"e34199b8-d8f7-4c47-92af-487e03848425","Type":"ContainerStarted","Data":"60e40b7799eaa8743120d2bf4604cbc803503eba2bf733daf96164edaba1b26b"}
Sep 04 23:41:12.055121 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:12.055081    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:12.155145 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:12.155098    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:12.331605 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:12.331527    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rq9hz" event={"ID":"e34199b8-d8f7-4c47-92af-487e03848425","Type":"ContainerStarted","Data":"5efdacbb1dac55123222a86a768bb44c0ae8d3129dc95c9546a94ada465f9c83"}
Sep 04 23:41:12.347762 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:12.347717    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rq9hz" podStartSLOduration=1.360676961 podStartE2EDuration="4.347703811s" totalImagesPullingTime="2.98702685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:41:12.346059088 +0000 UTC m=+260.594059604" watchObservedRunningTime="2026-09-04 23:41:12.347703811 +0000 UTC m=+260.595704326"
Sep 04 23:41:16.824198 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:16.824108    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:16.924412 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:16.924375    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:19.074328 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:19.074286    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:24.245914 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:24.245849    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:24.308159 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:24.308128    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:35.375870 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.375825    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:35.424406 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.424375    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"]
Sep 04 23:41:35.427543 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.427527    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:35.430344 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.430323    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 04 23:41:35.431764 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.431728    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 04 23:41:35.431764 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.431753    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 04 23:41:35.431917 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.431805    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 04 23:41:35.431917 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.431860    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jbpgd\""
Sep 04 23:41:35.432054 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.432040    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 04 23:41:35.436696 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.436674    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"]
Sep 04 23:41:35.475438 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.475412    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:35.557097 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.557075    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:35.562638 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.562620    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjn2h\" (UniqueName: \"kubernetes.io/projected/10d1c54a-cee6-4fa3-8c35-a6a022a8b315-kube-api-access-vjn2h\") pod \"lws-controller-manager-58b866555c-g6ls7\" (UID: \"10d1c54a-cee6-4fa3-8c35-a6a022a8b315\") " pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:35.562710 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.562667    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10d1c54a-cee6-4fa3-8c35-a6a022a8b315-cert\") pod \"lws-controller-manager-58b866555c-g6ls7\" (UID: \"10d1c54a-cee6-4fa3-8c35-a6a022a8b315\") " pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:35.562747 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.562706    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/10d1c54a-cee6-4fa3-8c35-a6a022a8b315-metrics-cert\") pod \"lws-controller-manager-58b866555c-g6ls7\" (UID: \"10d1c54a-cee6-4fa3-8c35-a6a022a8b315\") " pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:35.562747 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.562732    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/10d1c54a-cee6-4fa3-8c35-a6a022a8b315-manager-config\") pod \"lws-controller-manager-58b866555c-g6ls7\" (UID: \"10d1c54a-cee6-4fa3-8c35-a6a022a8b315\") " pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:35.663522 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.663475    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjn2h\" (UniqueName: \"kubernetes.io/projected/10d1c54a-cee6-4fa3-8c35-a6a022a8b315-kube-api-access-vjn2h\") pod \"lws-controller-manager-58b866555c-g6ls7\" (UID: \"10d1c54a-cee6-4fa3-8c35-a6a022a8b315\") " pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:35.663582 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.663523    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10d1c54a-cee6-4fa3-8c35-a6a022a8b315-cert\") pod \"lws-controller-manager-58b866555c-g6ls7\" (UID: \"10d1c54a-cee6-4fa3-8c35-a6a022a8b315\") " pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:35.663582 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.663552    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/10d1c54a-cee6-4fa3-8c35-a6a022a8b315-metrics-cert\") pod \"lws-controller-manager-58b866555c-g6ls7\" (UID: \"10d1c54a-cee6-4fa3-8c35-a6a022a8b315\") " pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:35.663582 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.663571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/10d1c54a-cee6-4fa3-8c35-a6a022a8b315-manager-config\") pod \"lws-controller-manager-58b866555c-g6ls7\" (UID: \"10d1c54a-cee6-4fa3-8c35-a6a022a8b315\") " pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:35.664165 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.664140    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/10d1c54a-cee6-4fa3-8c35-a6a022a8b315-manager-config\") pod \"lws-controller-manager-58b866555c-g6ls7\" (UID: \"10d1c54a-cee6-4fa3-8c35-a6a022a8b315\") " pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:35.666288 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.666261    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/10d1c54a-cee6-4fa3-8c35-a6a022a8b315-metrics-cert\") pod \"lws-controller-manager-58b866555c-g6ls7\" (UID: \"10d1c54a-cee6-4fa3-8c35-a6a022a8b315\") " pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:35.666381 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.666295    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10d1c54a-cee6-4fa3-8c35-a6a022a8b315-cert\") pod \"lws-controller-manager-58b866555c-g6ls7\" (UID: \"10d1c54a-cee6-4fa3-8c35-a6a022a8b315\") " pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:35.676757 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.676734    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjn2h\" (UniqueName: \"kubernetes.io/projected/10d1c54a-cee6-4fa3-8c35-a6a022a8b315-kube-api-access-vjn2h\") pod \"lws-controller-manager-58b866555c-g6ls7\" (UID: \"10d1c54a-cee6-4fa3-8c35-a6a022a8b315\") " pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:35.737343 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:35.737322    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:36.009344 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.009322    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"]
Sep 04 23:41:36.011857 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:41:36.011828    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d1c54a_cee6_4fa3_8c35_a6a022a8b315.slice/crio-6bcf70644accf8f5f0321fa25badd7b9d0c508bdfb5b5d32cee3f5e58e30daa4 WatchSource:0}: Error finding container 6bcf70644accf8f5f0321fa25badd7b9d0c508bdfb5b5d32cee3f5e58e30daa4: Status 404 returned error can't find the container with id 6bcf70644accf8f5f0321fa25badd7b9d0c508bdfb5b5d32cee3f5e58e30daa4
Sep 04 23:41:36.058163 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.058131    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:36.349115 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.349035    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:36.421840 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.421810    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7" event={"ID":"10d1c54a-cee6-4fa3-8c35-a6a022a8b315","Type":"ContainerStarted","Data":"6bcf70644accf8f5f0321fa25badd7b9d0c508bdfb5b5d32cee3f5e58e30daa4"}
Sep 04 23:41:36.441415 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.441377    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:36.474806 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.473543    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"]
Sep 04 23:41:36.478842 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.478821    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.481722 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.481702    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 04 23:41:36.481834 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.481707    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mm9nl\""
Sep 04 23:41:36.481834 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.481763    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 04 23:41:36.481834 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.481819    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 04 23:41:36.481985 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.481960    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 04 23:41:36.500196 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.500177    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"]
Sep 04 23:41:36.570655 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.570629    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-tmp\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.570739 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.570668    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.570739 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.570695    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.570809 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.570762    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9k4v\" (UniqueName: \"kubernetes.io/projected/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-kube-api-access-z9k4v\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.570892 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.570806    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-manifests\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.595436 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.595415    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:36.671591 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.671566    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.671702 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.671619    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.671702 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.671680    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9k4v\" (UniqueName: \"kubernetes.io/projected/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-kube-api-access-z9k4v\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.671818 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.671725    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-manifests\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.671818 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.671764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-tmp\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.672170 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.672144    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-manifests\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.674060 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.674039    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-tmp\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.674302 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.674284    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.674456 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.674438    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.684376 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.684353    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9k4v\" (UniqueName: \"kubernetes.io/projected/6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6-kube-api-access-z9k4v\") pod \"opendatahub-operator-controller-manager-6b9496bc86-krnwn\" (UID: \"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6\") " pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.696672 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.696640    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:36.788742 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.788719    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:36.799942 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.799818    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:36.896760 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.896714    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:36.997407 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:36.997176    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:37.085041 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:37.084715    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"]
Sep 04 23:41:37.086884 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:37.086855    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:37.087611 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:41:37.087582    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad6cdca_685c_4ec5_acbf_b1d9a86c2ed6.slice/crio-2b5fe5409a464e85807dfa04554492ebaa255d3ae64adc1645880c5f3ae3ddea WatchSource:0}: Error finding container 2b5fe5409a464e85807dfa04554492ebaa255d3ae64adc1645880c5f3ae3ddea: Status 404 returned error can't find the container with id 2b5fe5409a464e85807dfa04554492ebaa255d3ae64adc1645880c5f3ae3ddea
Sep 04 23:41:37.428859 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:37.428824    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn" event={"ID":"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6","Type":"ContainerStarted","Data":"2b5fe5409a464e85807dfa04554492ebaa255d3ae64adc1645880c5f3ae3ddea"}
Sep 04 23:41:39.439599 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:39.439565    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7" event={"ID":"10d1c54a-cee6-4fa3-8c35-a6a022a8b315","Type":"ContainerStarted","Data":"52e66842b975afa8945dc46b02bc1300de8a73f4b3125de9440f4aa25d277f69"}
Sep 04 23:41:39.440053 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:39.439656    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:39.459579 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:39.459532    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7" podStartSLOduration=2.069385915 podStartE2EDuration="4.459516871s" totalImagesPullingTime="2.390130956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:41:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:41:39.458996309 +0000 UTC m=+287.706996849" watchObservedRunningTime="2026-09-04 23:41:39.459516871 +0000 UTC m=+287.707517386"
Sep 04 23:41:41.449053 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:41.449016    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn" event={"ID":"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6","Type":"ContainerStarted","Data":"111370d1ee8f2fefbb74bdde7395d9946e253a6776c015eb662cc5aa8d54ffa6"}
Sep 04 23:41:42.454417 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:42.454384    2564 generic.go:356] "Generic (PLEG): container finished" podID="6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6" containerID="111370d1ee8f2fefbb74bdde7395d9946e253a6776c015eb662cc5aa8d54ffa6" exitCode=0
Sep 04 23:41:42.454884 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:42.454430    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn" event={"ID":"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6","Type":"ContainerDied","Data":"111370d1ee8f2fefbb74bdde7395d9946e253a6776c015eb662cc5aa8d54ffa6"}
Sep 04 23:41:43.460976 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:43.460939    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn" event={"ID":"6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6","Type":"ContainerStarted","Data":"746ae3c8cee36b6f5f1364ae33775e4ccbde2e086c4b7da8f1de4779863e9169"}
Sep 04 23:41:43.461410 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:43.461062    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:43.485053 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:43.484998    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn" podStartSLOduration=2.3583486320000002 podStartE2EDuration="7.484983189s" totalImagesPullingTime="4.050798592s" totalInitContainerRuntime="1.075835965s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:41:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:41:43.482475097 +0000 UTC m=+291.730475633" watchObservedRunningTime="2026-09-04 23:41:43.484983189 +0000 UTC m=+291.732983704"
Sep 04 23:41:48.172146 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:48.172109    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:48.272891 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:48.272850    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:50.446519 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:50.446490    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58b866555c-g6ls7"
Sep 04 23:41:52.101205 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:52.101179    2564 kubelet.go:1648] "Image garbage collection succeeded"
Sep 04 23:41:54.466729 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:54.466698    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b9496bc86-krnwn"
Sep 04 23:41:56.019817 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.019777    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:56.115025 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.114995    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:56.355957 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.355860    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:56.417145 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.417118    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"]
Sep 04 23:41:56.420211 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.420196    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"
Sep 04 23:41:56.423304 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.423271    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 04 23:41:56.423304 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.423297    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-fmcfs\""
Sep 04 23:41:56.423464 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.423309    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 04 23:41:56.423640 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.423626    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 04 23:41:56.423689 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.423657    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 04 23:41:56.434963 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.434932    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"]
Sep 04 23:41:56.520612 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.520585    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71b6a21b-ccbc-465d-80e0-b9d235f75a3d-tmp\") pod \"kube-auth-proxy-5d989d9b5-nppzf\" (UID: \"71b6a21b-ccbc-465d-80e0-b9d235f75a3d\") " pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"
Sep 04 23:41:56.520719 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.520616    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b9fx\" (UniqueName: \"kubernetes.io/projected/71b6a21b-ccbc-465d-80e0-b9d235f75a3d-kube-api-access-7b9fx\") pod \"kube-auth-proxy-5d989d9b5-nppzf\" (UID: \"71b6a21b-ccbc-465d-80e0-b9d235f75a3d\") " pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"
Sep 04 23:41:56.520719 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.520685    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71b6a21b-ccbc-465d-80e0-b9d235f75a3d-tls-certs\") pod \"kube-auth-proxy-5d989d9b5-nppzf\" (UID: \"71b6a21b-ccbc-465d-80e0-b9d235f75a3d\") " pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"
Sep 04 23:41:56.622018 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.621990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71b6a21b-ccbc-465d-80e0-b9d235f75a3d-tmp\") pod \"kube-auth-proxy-5d989d9b5-nppzf\" (UID: \"71b6a21b-ccbc-465d-80e0-b9d235f75a3d\") " pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"
Sep 04 23:41:56.622124 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.622021    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b9fx\" (UniqueName: \"kubernetes.io/projected/71b6a21b-ccbc-465d-80e0-b9d235f75a3d-kube-api-access-7b9fx\") pod \"kube-auth-proxy-5d989d9b5-nppzf\" (UID: \"71b6a21b-ccbc-465d-80e0-b9d235f75a3d\") " pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"
Sep 04 23:41:56.622124 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.622058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71b6a21b-ccbc-465d-80e0-b9d235f75a3d-tls-certs\") pod \"kube-auth-proxy-5d989d9b5-nppzf\" (UID: \"71b6a21b-ccbc-465d-80e0-b9d235f75a3d\") " pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"
Sep 04 23:41:56.624614 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.624586    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71b6a21b-ccbc-465d-80e0-b9d235f75a3d-tmp\") pod \"kube-auth-proxy-5d989d9b5-nppzf\" (UID: \"71b6a21b-ccbc-465d-80e0-b9d235f75a3d\") " pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"
Sep 04 23:41:56.624840 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.624821    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71b6a21b-ccbc-465d-80e0-b9d235f75a3d-tls-certs\") pod \"kube-auth-proxy-5d989d9b5-nppzf\" (UID: \"71b6a21b-ccbc-465d-80e0-b9d235f75a3d\") " pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"
Sep 04 23:41:56.630160 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.630142    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b9fx\" (UniqueName: \"kubernetes.io/projected/71b6a21b-ccbc-465d-80e0-b9d235f75a3d-kube-api-access-7b9fx\") pod \"kube-auth-proxy-5d989d9b5-nppzf\" (UID: \"71b6a21b-ccbc-465d-80e0-b9d235f75a3d\") " pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"
Sep 04 23:41:56.668186 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.668159    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:56.729559 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.729539    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"
Sep 04 23:41:56.999147 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:56.999027    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf"]
Sep 04 23:41:57.002190 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:41:57.002168    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b6a21b_ccbc_465d_80e0_b9d235f75a3d.slice/crio-f8a700093b0a8f51600ab6a913b0a9ea50f539e5a6c6b439e4bb1767b310647a WatchSource:0}: Error finding container f8a700093b0a8f51600ab6a913b0a9ea50f539e5a6c6b439e4bb1767b310647a: Status 404 returned error can't find the container with id f8a700093b0a8f51600ab6a913b0a9ea50f539e5a6c6b439e4bb1767b310647a
Sep 04 23:41:57.004464 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:57.004449    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 23:41:57.514310 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:57.514276    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf" event={"ID":"71b6a21b-ccbc-465d-80e0-b9d235f75a3d","Type":"ContainerStarted","Data":"f8a700093b0a8f51600ab6a913b0a9ea50f539e5a6c6b439e4bb1767b310647a"}
Sep 04 23:41:57.647717 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:57.647682    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:57.704737 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:57.704676    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:58.622738 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:58.622669    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:59.496455 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:59.496422    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:41:59.553499 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:41:59.553460    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:00.529226 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:00.529185    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf" event={"ID":"71b6a21b-ccbc-465d-80e0-b9d235f75a3d","Type":"ContainerStarted","Data":"ac49bf89b8c1f9dfb30554687b920e435811f15d0f1545a31888972a1bb712ac"}
Sep 04 23:42:00.545432 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:00.545388    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5d989d9b5-nppzf" podStartSLOduration=1.812600475 podStartE2EDuration="4.545373639s" totalImagesPullingTime="2.732773164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:41:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:42:00.544241091 +0000 UTC m=+308.792241627" watchObservedRunningTime="2026-09-04 23:42:00.545373639 +0000 UTC m=+308.793374155"
Sep 04 23:42:02.111963 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:02.111912    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:02.173165 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:02.173129    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:02.683352 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:02.683325    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:02.804126 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:02.804098    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:02.903880 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:02.903850    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:07.262270 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:07.262204    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:07.362647 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:07.362609    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:07.462834 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:07.462808    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:07.692736 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:07.692692    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:07.758135 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:07.758108    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:07.900834 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:07.900803    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:08.154408 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:08.154383    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:08.209347 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:08.209326    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:16.922075 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:16.922037    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:17.023763 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.023726    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c"]
Sep 04 23:42:17.027092 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.027071    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c"
Sep 04 23:42:17.029995 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.029968    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-jww77\""
Sep 04 23:42:17.030109 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.029969    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 04 23:42:17.037819 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.037799    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c"]
Sep 04 23:42:17.053293 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.053264    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:17.080854 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.080832    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11cb0f47-96ed-49e5-9607-9934a2a3dabb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qx95c\" (UID: \"11cb0f47-96ed-49e5-9607-9934a2a3dabb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c"
Sep 04 23:42:17.080983 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.080870    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzx5g\" (UniqueName: \"kubernetes.io/projected/11cb0f47-96ed-49e5-9607-9934a2a3dabb-kube-api-access-mzx5g\") pod \"llmisvc-controller-manager-777c6bd7d5-qx95c\" (UID: \"11cb0f47-96ed-49e5-9607-9934a2a3dabb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c"
Sep 04 23:42:17.149180 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.149148    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:17.181400 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.181348    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11cb0f47-96ed-49e5-9607-9934a2a3dabb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qx95c\" (UID: \"11cb0f47-96ed-49e5-9607-9934a2a3dabb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c"
Sep 04 23:42:17.181400 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.181385    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzx5g\" (UniqueName: \"kubernetes.io/projected/11cb0f47-96ed-49e5-9607-9934a2a3dabb-kube-api-access-mzx5g\") pod \"llmisvc-controller-manager-777c6bd7d5-qx95c\" (UID: \"11cb0f47-96ed-49e5-9607-9934a2a3dabb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c"
Sep 04 23:42:17.183910 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.183877    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11cb0f47-96ed-49e5-9607-9934a2a3dabb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qx95c\" (UID: \"11cb0f47-96ed-49e5-9607-9934a2a3dabb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c"
Sep 04 23:42:17.193926 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.191084    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzx5g\" (UniqueName: \"kubernetes.io/projected/11cb0f47-96ed-49e5-9607-9934a2a3dabb-kube-api-access-mzx5g\") pod \"llmisvc-controller-manager-777c6bd7d5-qx95c\" (UID: \"11cb0f47-96ed-49e5-9607-9934a2a3dabb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c"
Sep 04 23:42:17.250534 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.250513    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:17.337667 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.337643    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c"
Sep 04 23:42:17.605032 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:17.604943    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c"]
Sep 04 23:42:17.607788 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:42:17.607761    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod11cb0f47_96ed_49e5_9607_9934a2a3dabb.slice/crio-a4cbe09a5eee79dfddd582139e3b6e7027f6d2de27eff5c9fba9f927fdbd82ae WatchSource:0}: Error finding container a4cbe09a5eee79dfddd582139e3b6e7027f6d2de27eff5c9fba9f927fdbd82ae: Status 404 returned error can't find the container with id a4cbe09a5eee79dfddd582139e3b6e7027f6d2de27eff5c9fba9f927fdbd82ae
Sep 04 23:42:18.209557 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:18.209527    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:18.305921 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:18.305803    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:18.408024 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:18.407993    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:18.602544 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:18.602462    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c" event={"ID":"11cb0f47-96ed-49e5-9607-9934a2a3dabb","Type":"ContainerStarted","Data":"a4cbe09a5eee79dfddd582139e3b6e7027f6d2de27eff5c9fba9f927fdbd82ae"}
Sep 04 23:42:20.613231 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:20.613195    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c" event={"ID":"11cb0f47-96ed-49e5-9607-9934a2a3dabb","Type":"ContainerStarted","Data":"de0ae3829f7e2cf0db832eb1eb1a00733ff73fe7491b4b3cbd2eaf347e7b9320"}
Sep 04 23:42:20.613650 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:20.613261    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c"
Sep 04 23:42:20.628915 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:20.628850    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c" podStartSLOduration=1.233775491 podStartE2EDuration="3.628835241s" totalImagesPullingTime="2.39505975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:42:20.627200262 +0000 UTC m=+328.875200815" watchObservedRunningTime="2026-09-04 23:42:20.628835241 +0000 UTC m=+328.876835757"
Sep 04 23:42:23.888642 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:23.888595    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:23.950803 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:23.950769    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:42:51.618828 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:42:51.618744    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qx95c"
Sep 04 23:43:00.636497 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.636464    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86d4c4dc55-z4mcm"]
Sep 04 23:43:00.650864 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.650838    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86d4c4dc55-z4mcm"]
Sep 04 23:43:00.651024 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.650953    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.684453 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.684420    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:00.735701 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.735674    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:00.806652 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.806633    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba87952b-527d-4f5a-959c-dd1667ee7188-service-ca\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.806762 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.806664    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba87952b-527d-4f5a-959c-dd1667ee7188-oauth-serving-cert\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.806762 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.806693    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba87952b-527d-4f5a-959c-dd1667ee7188-console-config\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.806762 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.806721    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba87952b-527d-4f5a-959c-dd1667ee7188-console-serving-cert\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.806762 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.806747    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5695b\" (UniqueName: \"kubernetes.io/projected/ba87952b-527d-4f5a-959c-dd1667ee7188-kube-api-access-5695b\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.806890 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.806770    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba87952b-527d-4f5a-959c-dd1667ee7188-console-oauth-config\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.806890 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.806845    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba87952b-527d-4f5a-959c-dd1667ee7188-trusted-ca-bundle\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.907678 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.907622    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba87952b-527d-4f5a-959c-dd1667ee7188-console-config\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.907678 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.907650    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba87952b-527d-4f5a-959c-dd1667ee7188-console-serving-cert\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.907678 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.907673    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5695b\" (UniqueName: \"kubernetes.io/projected/ba87952b-527d-4f5a-959c-dd1667ee7188-kube-api-access-5695b\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.907817 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.907798    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba87952b-527d-4f5a-959c-dd1667ee7188-console-oauth-config\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.907868 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.907850    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba87952b-527d-4f5a-959c-dd1667ee7188-trusted-ca-bundle\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.907936 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.907914    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba87952b-527d-4f5a-959c-dd1667ee7188-service-ca\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.907993 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.907948    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba87952b-527d-4f5a-959c-dd1667ee7188-oauth-serving-cert\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.908481 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.908451    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ba87952b-527d-4f5a-959c-dd1667ee7188-console-config\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.908580 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.908527    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ba87952b-527d-4f5a-959c-dd1667ee7188-oauth-serving-cert\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.908639 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.908621    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ba87952b-527d-4f5a-959c-dd1667ee7188-service-ca\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.909277 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.909221    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba87952b-527d-4f5a-959c-dd1667ee7188-trusted-ca-bundle\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.910571 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.910543    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ba87952b-527d-4f5a-959c-dd1667ee7188-console-oauth-config\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.910676 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.910658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba87952b-527d-4f5a-959c-dd1667ee7188-console-serving-cert\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.915591 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.915569    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5695b\" (UniqueName: \"kubernetes.io/projected/ba87952b-527d-4f5a-959c-dd1667ee7188-kube-api-access-5695b\") pod \"console-86d4c4dc55-z4mcm\" (UID: \"ba87952b-527d-4f5a-959c-dd1667ee7188\") " pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:00.960521 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:00.960498    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:01.229688 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:01.229662    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86d4c4dc55-z4mcm"]
Sep 04 23:43:01.232401 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:43:01.232370    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba87952b_527d_4f5a_959c_dd1667ee7188.slice/crio-f0540573fead084893669f0ff4467cb0a488d040ba1d5b67b3f68a0500f09fa5 WatchSource:0}: Error finding container f0540573fead084893669f0ff4467cb0a488d040ba1d5b67b3f68a0500f09fa5: Status 404 returned error can't find the container with id f0540573fead084893669f0ff4467cb0a488d040ba1d5b67b3f68a0500f09fa5
Sep 04 23:43:01.791945 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:01.791910    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d4c4dc55-z4mcm" event={"ID":"ba87952b-527d-4f5a-959c-dd1667ee7188","Type":"ContainerStarted","Data":"616f231ee826cead484a2a001d772606bc2956d7d55f6894ea35e90814f9f181"}
Sep 04 23:43:01.792310 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:01.791968    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d4c4dc55-z4mcm" event={"ID":"ba87952b-527d-4f5a-959c-dd1667ee7188","Type":"ContainerStarted","Data":"f0540573fead084893669f0ff4467cb0a488d040ba1d5b67b3f68a0500f09fa5"}
Sep 04 23:43:01.808786 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:01.808736    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86d4c4dc55-z4mcm" podStartSLOduration=1.808721436 podStartE2EDuration="1.808721436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:43:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:43:01.807737136 +0000 UTC m=+370.055737667" watchObservedRunningTime="2026-09-04 23:43:01.808721436 +0000 UTC m=+370.056721952"
Sep 04 23:43:05.286586 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:05.286540    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:05.387092 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:05.387046    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:08.083252 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:08.083217    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:08.187704 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:08.187673    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:10.961513 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:10.961474    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:10.961877 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:10.961553    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:10.966271 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:10.966250    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:11.837326 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:11.837296    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86d4c4dc55-z4mcm"
Sep 04 23:43:11.888428 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:11.888391    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68f776d88d-s226f"]
Sep 04 23:43:11.988609 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:11.988573    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:12.088429 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:12.088352    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:12.190883 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:12.190852    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:12.293030 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:12.293009    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:15.832292 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:15.832259    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:15.907297 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:15.907249    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:19.924098 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:19.924053    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:20.719236 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:20.719205    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:21.515948 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:21.515890    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:24.519579 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:24.519541    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:24.843171 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:24.843089    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:24.955568 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:24.955532    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:25.052426 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:25.052394    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:26.374711 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:26.374675    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:26.472667 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:26.472639    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:27.528944 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:27.528891    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:28.554301 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:28.554266    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:29.739821 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:29.739789    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:31.165959 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:31.165926    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:31.872745 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:31.872704    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:32.187083 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:32.186989    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:35.128087 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:35.128047    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:35.228123 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:35.228090    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:35.525372 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:35.525288    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:35.582343 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:35.582316    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:35.928914 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:35.928864    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:35.992151 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:35.992120    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:36.915257 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:36.915212    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68f776d88d-s226f" podUID="01b597dd-b90d-46d5-903c-d6760312794d" containerName="console" containerID="cri-o://ec10b47a99033ef9b56ab6964319303ac39e0b3bc9266a736a1e2a973eaf481e" gracePeriod=15
Sep 04 23:43:37.153586 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.153562    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:43:37.185976 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.185905    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-trusted-ca-bundle\") pod \"01b597dd-b90d-46d5-903c-d6760312794d\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") "
Sep 04 23:43:37.185976 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.185971    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01b597dd-b90d-46d5-903c-d6760312794d-console-oauth-config\") pod \"01b597dd-b90d-46d5-903c-d6760312794d\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") "
Sep 04 23:43:37.186175 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.185995    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-service-ca\") pod \"01b597dd-b90d-46d5-903c-d6760312794d\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") "
Sep 04 23:43:37.186175 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.186010    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01b597dd-b90d-46d5-903c-d6760312794d-console-serving-cert\") pod \"01b597dd-b90d-46d5-903c-d6760312794d\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") "
Sep 04 23:43:37.186175 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.186034    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-oauth-serving-cert\") pod \"01b597dd-b90d-46d5-903c-d6760312794d\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") "
Sep 04 23:43:37.186328 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.186228    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nplrp\" (UniqueName: \"kubernetes.io/projected/01b597dd-b90d-46d5-903c-d6760312794d-kube-api-access-nplrp\") pod \"01b597dd-b90d-46d5-903c-d6760312794d\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") "
Sep 04 23:43:37.186328 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.186277    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-console-config\") pod \"01b597dd-b90d-46d5-903c-d6760312794d\" (UID: \"01b597dd-b90d-46d5-903c-d6760312794d\") "
Sep 04 23:43:37.186434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.186226    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "01b597dd-b90d-46d5-903c-d6760312794d" (UID: "01b597dd-b90d-46d5-903c-d6760312794d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:43:37.186434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.186414    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "01b597dd-b90d-46d5-903c-d6760312794d" (UID: "01b597dd-b90d-46d5-903c-d6760312794d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:43:37.186546 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.186424    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-service-ca" (OuterVolumeSpecName: "service-ca") pod "01b597dd-b90d-46d5-903c-d6760312794d" (UID: "01b597dd-b90d-46d5-903c-d6760312794d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:43:37.186625 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.186606    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-trusted-ca-bundle\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:43:37.186684 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.186629    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:43:37.186684 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.186642    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-oauth-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:43:37.186974 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.186741    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-console-config" (OuterVolumeSpecName: "console-config") pod "01b597dd-b90d-46d5-903c-d6760312794d" (UID: "01b597dd-b90d-46d5-903c-d6760312794d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:43:37.188322 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.188295    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01b597dd-b90d-46d5-903c-d6760312794d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "01b597dd-b90d-46d5-903c-d6760312794d" (UID: "01b597dd-b90d-46d5-903c-d6760312794d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:43:37.188502 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.188474    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01b597dd-b90d-46d5-903c-d6760312794d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "01b597dd-b90d-46d5-903c-d6760312794d" (UID: "01b597dd-b90d-46d5-903c-d6760312794d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:43:37.188597 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.188501    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01b597dd-b90d-46d5-903c-d6760312794d-kube-api-access-nplrp" (OuterVolumeSpecName: "kube-api-access-nplrp") pod "01b597dd-b90d-46d5-903c-d6760312794d" (UID: "01b597dd-b90d-46d5-903c-d6760312794d"). InnerVolumeSpecName "kube-api-access-nplrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:43:37.287626 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.287593    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01b597dd-b90d-46d5-903c-d6760312794d-console-oauth-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:43:37.287626 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.287618    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01b597dd-b90d-46d5-903c-d6760312794d-console-serving-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:43:37.287626 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.287628    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nplrp\" (UniqueName: \"kubernetes.io/projected/01b597dd-b90d-46d5-903c-d6760312794d-kube-api-access-nplrp\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:43:37.287836 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.287638    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01b597dd-b90d-46d5-903c-d6760312794d-console-config\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:43:37.949683 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.949645    2564 generic.go:356] "Generic (PLEG): container finished" podID="01b597dd-b90d-46d5-903c-d6760312794d" containerID="ec10b47a99033ef9b56ab6964319303ac39e0b3bc9266a736a1e2a973eaf481e" exitCode=0
Sep 04 23:43:37.950075 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.949690    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f776d88d-s226f" event={"ID":"01b597dd-b90d-46d5-903c-d6760312794d","Type":"ContainerDied","Data":"ec10b47a99033ef9b56ab6964319303ac39e0b3bc9266a736a1e2a973eaf481e"}
Sep 04 23:43:37.950075 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.949709    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f776d88d-s226f"
Sep 04 23:43:37.950075 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.949741    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f776d88d-s226f" event={"ID":"01b597dd-b90d-46d5-903c-d6760312794d","Type":"ContainerDied","Data":"c7a9dd3d79da6a95de27b5fd3a6683df5f35bef58c834a001486a17c6bc43136"}
Sep 04 23:43:37.950075 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.949767    2564 scope.go:117] "RemoveContainer" containerID="ec10b47a99033ef9b56ab6964319303ac39e0b3bc9266a736a1e2a973eaf481e"
Sep 04 23:43:37.958308 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.958292    2564 scope.go:117] "RemoveContainer" containerID="ec10b47a99033ef9b56ab6964319303ac39e0b3bc9266a736a1e2a973eaf481e"
Sep 04 23:43:37.958575 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:43:37.958559    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec10b47a99033ef9b56ab6964319303ac39e0b3bc9266a736a1e2a973eaf481e\": container with ID starting with ec10b47a99033ef9b56ab6964319303ac39e0b3bc9266a736a1e2a973eaf481e not found: ID does not exist" containerID="ec10b47a99033ef9b56ab6964319303ac39e0b3bc9266a736a1e2a973eaf481e"
Sep 04 23:43:37.958615 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.958581    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec10b47a99033ef9b56ab6964319303ac39e0b3bc9266a736a1e2a973eaf481e"} err="failed to get container status \"ec10b47a99033ef9b56ab6964319303ac39e0b3bc9266a736a1e2a973eaf481e\": rpc error: code = NotFound desc = could not find container \"ec10b47a99033ef9b56ab6964319303ac39e0b3bc9266a736a1e2a973eaf481e\": container with ID starting with ec10b47a99033ef9b56ab6964319303ac39e0b3bc9266a736a1e2a973eaf481e not found: ID does not exist"
Sep 04 23:43:37.969751 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.969732    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68f776d88d-s226f"]
Sep 04 23:43:37.974055 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:37.974034    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68f776d88d-s226f"]
Sep 04 23:43:38.173631 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:38.173605    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01b597dd-b90d-46d5-903c-d6760312794d" path="/var/lib/kubelet/pods/01b597dd-b90d-46d5-903c-d6760312794d/volumes"
Sep 04 23:43:43.382461 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.382432    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"]
Sep 04 23:43:43.382871 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.382771    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01b597dd-b90d-46d5-903c-d6760312794d" containerName="console"
Sep 04 23:43:43.382871 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.382782    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="01b597dd-b90d-46d5-903c-d6760312794d" containerName="console"
Sep 04 23:43:43.382871 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.382847    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="01b597dd-b90d-46d5-903c-d6760312794d" containerName="console"
Sep 04 23:43:43.387118 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.387102    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"
Sep 04 23:43:43.389788 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.389765    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 04 23:43:43.389788 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.389765    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 04 23:43:43.389989 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.389844    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j28nh\""
Sep 04 23:43:43.400209 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.400188    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"]
Sep 04 23:43:43.436641 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.436614    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2n4k\" (UniqueName: \"kubernetes.io/projected/042f366d-62d3-4f8d-bcbc-cb1a272a716c-kube-api-access-c2n4k\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-9wgwq\" (UID: \"042f366d-62d3-4f8d-bcbc-cb1a272a716c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"
Sep 04 23:43:43.436737 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.436660    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/042f366d-62d3-4f8d-bcbc-cb1a272a716c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-9wgwq\" (UID: \"042f366d-62d3-4f8d-bcbc-cb1a272a716c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"
Sep 04 23:43:43.537236 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.537213    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2n4k\" (UniqueName: \"kubernetes.io/projected/042f366d-62d3-4f8d-bcbc-cb1a272a716c-kube-api-access-c2n4k\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-9wgwq\" (UID: \"042f366d-62d3-4f8d-bcbc-cb1a272a716c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"
Sep 04 23:43:43.537349 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.537258    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/042f366d-62d3-4f8d-bcbc-cb1a272a716c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-9wgwq\" (UID: \"042f366d-62d3-4f8d-bcbc-cb1a272a716c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"
Sep 04 23:43:43.537547 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.537532    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/042f366d-62d3-4f8d-bcbc-cb1a272a716c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-9wgwq\" (UID: \"042f366d-62d3-4f8d-bcbc-cb1a272a716c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"
Sep 04 23:43:43.545576 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.545558    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2n4k\" (UniqueName: \"kubernetes.io/projected/042f366d-62d3-4f8d-bcbc-cb1a272a716c-kube-api-access-c2n4k\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-9wgwq\" (UID: \"042f366d-62d3-4f8d-bcbc-cb1a272a716c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"
Sep 04 23:43:43.697556 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.697484    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"
Sep 04 23:43:43.954655 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.954576    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:43.971249 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:43.971222    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"]
Sep 04 23:43:43.974865 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:43:43.974840    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod042f366d_62d3_4f8d_bcbc_cb1a272a716c.slice/crio-88d2e0da94215af6858425a262626ea04bf415b569c7ffa35b28370797d8756f WatchSource:0}: Error finding container 88d2e0da94215af6858425a262626ea04bf415b569c7ffa35b28370797d8756f: Status 404 returned error can't find the container with id 88d2e0da94215af6858425a262626ea04bf415b569c7ffa35b28370797d8756f
Sep 04 23:43:44.062326 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:44.062299    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:44.985841 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:44.985801    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq" event={"ID":"042f366d-62d3-4f8d-bcbc-cb1a272a716c","Type":"ContainerStarted","Data":"88d2e0da94215af6858425a262626ea04bf415b569c7ffa35b28370797d8756f"}
Sep 04 23:43:46.940995 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:46.940961    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:47.042422 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:47.042384    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:48.001617 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:48.001580    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq" event={"ID":"042f366d-62d3-4f8d-bcbc-cb1a272a716c","Type":"ContainerStarted","Data":"f6f3685e10a59cfc73ae5914e3f09cd121ee16f06213543dd159b280fe1d5aab"}
Sep 04 23:43:48.002076 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:48.001699    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"
Sep 04 23:43:48.021990 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:48.021946    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq" podStartSLOduration=1.45381212 podStartE2EDuration="5.021930874s" totalImagesPullingTime="3.568118754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:43:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:43:48.020286328 +0000 UTC m=+416.268286863" watchObservedRunningTime="2026-09-04 23:43:48.021930874 +0000 UTC m=+416.269931391"
Sep 04 23:43:49.376295 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:49.376260    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:49.429613 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:49.429579    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:56.415073 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:56.415024    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:56.464778 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:56.464743    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:56.746914 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:56.746823    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:56.798350 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:56.798322    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:56.940091 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:56.940057    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:56.996434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:56.996408    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:43:59.009080 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:43:59.009048    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"
Sep 04 23:44:05.750366 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:05.750334    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:05.850915 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:05.850863    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:08.750248 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:08.750195    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:08.851927 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:08.851881    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:11.207182 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:11.207144    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:11.307002 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:11.306967    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:15.329480 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:15.329393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:15.526460 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:15.526420    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:19.639917 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:19.639867    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:19.739183 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:19.739145    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:19.794984 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:19.794955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:20.425549 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:20.425519    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-bl8bt"]
Sep 04 23:44:20.427881 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:20.427861    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-bl8bt"
Sep 04 23:44:20.430635 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:20.430611    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rx5zz\""
Sep 04 23:44:20.436007 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:20.435983    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bl8bt"]
Sep 04 23:44:20.467958 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:20.467937    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:20.518704 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:20.518678    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc2dr\" (UniqueName: \"kubernetes.io/projected/e7f351fc-93b0-47c8-99b3-d6ba9d4bce93-kube-api-access-qc2dr\") pod \"authorino-b54df94b4-bl8bt\" (UID: \"e7f351fc-93b0-47c8-99b3-d6ba9d4bce93\") " pod="kuadrant-system/authorino-b54df94b4-bl8bt"
Sep 04 23:44:20.549785 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:20.549753    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:20.619542 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:20.619522    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc2dr\" (UniqueName: \"kubernetes.io/projected/e7f351fc-93b0-47c8-99b3-d6ba9d4bce93-kube-api-access-qc2dr\") pod \"authorino-b54df94b4-bl8bt\" (UID: \"e7f351fc-93b0-47c8-99b3-d6ba9d4bce93\") " pod="kuadrant-system/authorino-b54df94b4-bl8bt"
Sep 04 23:44:20.627422 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:20.627402    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc2dr\" (UniqueName: \"kubernetes.io/projected/e7f351fc-93b0-47c8-99b3-d6ba9d4bce93-kube-api-access-qc2dr\") pod \"authorino-b54df94b4-bl8bt\" (UID: \"e7f351fc-93b0-47c8-99b3-d6ba9d4bce93\") " pod="kuadrant-system/authorino-b54df94b4-bl8bt"
Sep 04 23:44:20.737655 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:20.737603    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-bl8bt"
Sep 04 23:44:21.004881 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:21.004861    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bl8bt"]
Sep 04 23:44:21.007190 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:44:21.007163    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7f351fc_93b0_47c8_99b3_d6ba9d4bce93.slice/crio-39e7b85ea0b36aa9fbec811f9273bb4648e886a8e8c6bb20e4fb3a00b2b22e44 WatchSource:0}: Error finding container 39e7b85ea0b36aa9fbec811f9273bb4648e886a8e8c6bb20e4fb3a00b2b22e44: Status 404 returned error can't find the container with id 39e7b85ea0b36aa9fbec811f9273bb4648e886a8e8c6bb20e4fb3a00b2b22e44
Sep 04 23:44:21.147911 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:21.147869    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-bl8bt" event={"ID":"e7f351fc-93b0-47c8-99b3-d6ba9d4bce93","Type":"ContainerStarted","Data":"39e7b85ea0b36aa9fbec811f9273bb4648e886a8e8c6bb20e4fb3a00b2b22e44"}
Sep 04 23:44:23.703352 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:23.703322    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bl8bt"]
Sep 04 23:44:24.166333 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:24.166300    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-bl8bt" event={"ID":"e7f351fc-93b0-47c8-99b3-d6ba9d4bce93","Type":"ContainerStarted","Data":"58896367abcaa596a77e791a4a8be435d6720e80f94bf3765056010f16ebf0e7"}
Sep 04 23:44:24.180446 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:24.180398    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-bl8bt" podStartSLOduration=2.078185577 podStartE2EDuration="4.180381689s" totalImagesPullingTime="2.102196112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:44:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:44:24.180073418 +0000 UTC m=+452.428073934" watchObservedRunningTime="2026-09-04 23:44:24.180381689 +0000 UTC m=+452.428382206"
Sep 04 23:44:25.171660 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:25.171616    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-bl8bt" podUID="e7f351fc-93b0-47c8-99b3-d6ba9d4bce93" containerName="authorino" containerID="cri-o://58896367abcaa596a77e791a4a8be435d6720e80f94bf3765056010f16ebf0e7" gracePeriod=30
Sep 04 23:44:25.409338 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:25.409317    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-bl8bt"
Sep 04 23:44:25.559685 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:25.559611    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qc2dr\" (UniqueName: \"kubernetes.io/projected/e7f351fc-93b0-47c8-99b3-d6ba9d4bce93-kube-api-access-qc2dr\") pod \"e7f351fc-93b0-47c8-99b3-d6ba9d4bce93\" (UID: \"e7f351fc-93b0-47c8-99b3-d6ba9d4bce93\") "
Sep 04 23:44:25.561875 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:25.561854    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7f351fc-93b0-47c8-99b3-d6ba9d4bce93-kube-api-access-qc2dr" (OuterVolumeSpecName: "kube-api-access-qc2dr") pod "e7f351fc-93b0-47c8-99b3-d6ba9d4bce93" (UID: "e7f351fc-93b0-47c8-99b3-d6ba9d4bce93"). InnerVolumeSpecName "kube-api-access-qc2dr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:44:25.660800 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:25.660778    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qc2dr\" (UniqueName: \"kubernetes.io/projected/e7f351fc-93b0-47c8-99b3-d6ba9d4bce93-kube-api-access-qc2dr\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:44:26.176343 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:26.176315    2564 generic.go:356] "Generic (PLEG): container finished" podID="e7f351fc-93b0-47c8-99b3-d6ba9d4bce93" containerID="58896367abcaa596a77e791a4a8be435d6720e80f94bf3765056010f16ebf0e7" exitCode=0
Sep 04 23:44:26.176747 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:26.176363    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-bl8bt"
Sep 04 23:44:26.176747 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:26.176415    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-bl8bt" event={"ID":"e7f351fc-93b0-47c8-99b3-d6ba9d4bce93","Type":"ContainerDied","Data":"58896367abcaa596a77e791a4a8be435d6720e80f94bf3765056010f16ebf0e7"}
Sep 04 23:44:26.176747 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:26.176465    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-bl8bt" event={"ID":"e7f351fc-93b0-47c8-99b3-d6ba9d4bce93","Type":"ContainerDied","Data":"39e7b85ea0b36aa9fbec811f9273bb4648e886a8e8c6bb20e4fb3a00b2b22e44"}
Sep 04 23:44:26.176747 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:26.176489    2564 scope.go:117] "RemoveContainer" containerID="58896367abcaa596a77e791a4a8be435d6720e80f94bf3765056010f16ebf0e7"
Sep 04 23:44:26.185335 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:26.185299    2564 scope.go:117] "RemoveContainer" containerID="58896367abcaa596a77e791a4a8be435d6720e80f94bf3765056010f16ebf0e7"
Sep 04 23:44:26.185687 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:44:26.185666    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58896367abcaa596a77e791a4a8be435d6720e80f94bf3765056010f16ebf0e7\": container with ID starting with 58896367abcaa596a77e791a4a8be435d6720e80f94bf3765056010f16ebf0e7 not found: ID does not exist" containerID="58896367abcaa596a77e791a4a8be435d6720e80f94bf3765056010f16ebf0e7"
Sep 04 23:44:26.185772 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:26.185692    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58896367abcaa596a77e791a4a8be435d6720e80f94bf3765056010f16ebf0e7"} err="failed to get container status \"58896367abcaa596a77e791a4a8be435d6720e80f94bf3765056010f16ebf0e7\": rpc error: code = NotFound desc = could not find container \"58896367abcaa596a77e791a4a8be435d6720e80f94bf3765056010f16ebf0e7\": container with ID starting with 58896367abcaa596a77e791a4a8be435d6720e80f94bf3765056010f16ebf0e7 not found: ID does not exist"
Sep 04 23:44:26.196533 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:26.196514    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bl8bt"]
Sep 04 23:44:26.202324 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:26.202303    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-bl8bt"]
Sep 04 23:44:28.176893 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:28.176855    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7f351fc-93b0-47c8-99b3-d6ba9d4bce93" path="/var/lib/kubelet/pods/e7f351fc-93b0-47c8-99b3-d6ba9d4bce93/volumes"
Sep 04 23:44:30.347843 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:30.347810    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:35.680148 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:35.680092    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:35.748999 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:35.748965    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:36.095105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:36.095034    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:49.098585 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.098547    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:49.323403 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.323373    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:49.567594 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.567563    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f9d9dd6c7-lt888"]
Sep 04 23:44:49.567922 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.567885    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7f351fc-93b0-47c8-99b3-d6ba9d4bce93" containerName="authorino"
Sep 04 23:44:49.567922 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.567917    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7f351fc-93b0-47c8-99b3-d6ba9d4bce93" containerName="authorino"
Sep 04 23:44:49.568013 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.567981    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7f351fc-93b0-47c8-99b3-d6ba9d4bce93" containerName="authorino"
Sep 04 23:44:49.570949 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.570933    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f9d9dd6c7-lt888"
Sep 04 23:44:49.577642 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.577605    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 04 23:44:49.578174 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.578146    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rx5zz\""
Sep 04 23:44:49.582086 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.582062    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f9d9dd6c7-lt888"]
Sep 04 23:44:49.634120 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.634090    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e59d932-7594-443d-8058-00f1f6673968-tls-cert\") pod \"authorino-6f9d9dd6c7-lt888\" (UID: \"2e59d932-7594-443d-8058-00f1f6673968\") " pod="kuadrant-system/authorino-6f9d9dd6c7-lt888"
Sep 04 23:44:49.634241 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.634127    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvjkx\" (UniqueName: \"kubernetes.io/projected/2e59d932-7594-443d-8058-00f1f6673968-kube-api-access-tvjkx\") pod \"authorino-6f9d9dd6c7-lt888\" (UID: \"2e59d932-7594-443d-8058-00f1f6673968\") " pod="kuadrant-system/authorino-6f9d9dd6c7-lt888"
Sep 04 23:44:49.734585 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.734553    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvjkx\" (UniqueName: \"kubernetes.io/projected/2e59d932-7594-443d-8058-00f1f6673968-kube-api-access-tvjkx\") pod \"authorino-6f9d9dd6c7-lt888\" (UID: \"2e59d932-7594-443d-8058-00f1f6673968\") " pod="kuadrant-system/authorino-6f9d9dd6c7-lt888"
Sep 04 23:44:49.734689 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.734654    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e59d932-7594-443d-8058-00f1f6673968-tls-cert\") pod \"authorino-6f9d9dd6c7-lt888\" (UID: \"2e59d932-7594-443d-8058-00f1f6673968\") " pod="kuadrant-system/authorino-6f9d9dd6c7-lt888"
Sep 04 23:44:49.737271 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.737242    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e59d932-7594-443d-8058-00f1f6673968-tls-cert\") pod \"authorino-6f9d9dd6c7-lt888\" (UID: \"2e59d932-7594-443d-8058-00f1f6673968\") " pod="kuadrant-system/authorino-6f9d9dd6c7-lt888"
Sep 04 23:44:49.742481 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.742457    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvjkx\" (UniqueName: \"kubernetes.io/projected/2e59d932-7594-443d-8058-00f1f6673968-kube-api-access-tvjkx\") pod \"authorino-6f9d9dd6c7-lt888\" (UID: \"2e59d932-7594-443d-8058-00f1f6673968\") " pod="kuadrant-system/authorino-6f9d9dd6c7-lt888"
Sep 04 23:44:49.882425 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:49.882389    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f9d9dd6c7-lt888"
Sep 04 23:44:50.150019 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:50.149957    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f9d9dd6c7-lt888"]
Sep 04 23:44:50.152251 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:44:50.152223    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e59d932_7594_443d_8058_00f1f6673968.slice/crio-fda40ac6f6709ac5218a3b6b941618c64ca28198f20da78339f135ef270b10ec WatchSource:0}: Error finding container fda40ac6f6709ac5218a3b6b941618c64ca28198f20da78339f135ef270b10ec: Status 404 returned error can't find the container with id fda40ac6f6709ac5218a3b6b941618c64ca28198f20da78339f135ef270b10ec
Sep 04 23:44:50.278668 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:50.278635    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f9d9dd6c7-lt888" event={"ID":"2e59d932-7594-443d-8058-00f1f6673968","Type":"ContainerStarted","Data":"fda40ac6f6709ac5218a3b6b941618c64ca28198f20da78339f135ef270b10ec"}
Sep 04 23:44:51.285660 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:51.285624    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f9d9dd6c7-lt888" event={"ID":"2e59d932-7594-443d-8058-00f1f6673968","Type":"ContainerStarted","Data":"c7a97367667c89169fcb63b7b76260b014bbfa7c6ee042c638d9a830b71302a8"}
Sep 04 23:44:51.299998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:51.299947    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6f9d9dd6c7-lt888" podStartSLOduration=1.847773136 podStartE2EDuration="2.299929923s" totalImagesPullingTime="452.156787ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:44:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:44:51.299750877 +0000 UTC m=+479.547751392" watchObservedRunningTime="2026-09-04 23:44:51.299929923 +0000 UTC m=+479.547930439"
Sep 04 23:44:55.791585 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:55.791538    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:55.892752 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:55.892713    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:55.947145 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:55.947111    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:56.149513 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:56.149488    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:56.282177 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:56.282145    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:59.790451 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:59.790417    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:44:59.889811 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:44:59.889777    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:01.984712 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:01.984677    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:32.396817 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:32.396769    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:32.465056 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:32.465030    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:33.799586 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:33.799536    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:33.901634 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:33.901596    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:33.969160 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:33.969130    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:34.123862 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.123834    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:34.391553 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.391480    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:34.488386 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.488349    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:34.575066 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.575027    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np"]
Sep 04 23:45:34.579143 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.579117    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np"
Sep 04 23:45:34.586916 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.582120    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 04 23:45:34.586916 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.582535    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-j6v2z\""
Sep 04 23:45:34.591846 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.591819    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np"]
Sep 04 23:45:34.671225 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.671136    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksb7n\" (UniqueName: \"kubernetes.io/projected/e3ba54fc-16ee-4eee-9826-5cd838046f10-kube-api-access-ksb7n\") pod \"payload-pre-processing-59fbc47bf8-4r6np\" (UID: \"e3ba54fc-16ee-4eee-9826-5cd838046f10\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np"
Sep 04 23:45:34.671225 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.671200    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3ba54fc-16ee-4eee-9826-5cd838046f10-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-4r6np\" (UID: \"e3ba54fc-16ee-4eee-9826-5cd838046f10\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np"
Sep 04 23:45:34.772217 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.772179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksb7n\" (UniqueName: \"kubernetes.io/projected/e3ba54fc-16ee-4eee-9826-5cd838046f10-kube-api-access-ksb7n\") pod \"payload-pre-processing-59fbc47bf8-4r6np\" (UID: \"e3ba54fc-16ee-4eee-9826-5cd838046f10\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np"
Sep 04 23:45:34.772381 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.772228    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3ba54fc-16ee-4eee-9826-5cd838046f10-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-4r6np\" (UID: \"e3ba54fc-16ee-4eee-9826-5cd838046f10\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np"
Sep 04 23:45:34.773235 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.773208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3ba54fc-16ee-4eee-9826-5cd838046f10-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-4r6np\" (UID: \"e3ba54fc-16ee-4eee-9826-5cd838046f10\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np"
Sep 04 23:45:34.781860 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.781831    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksb7n\" (UniqueName: \"kubernetes.io/projected/e3ba54fc-16ee-4eee-9826-5cd838046f10-kube-api-access-ksb7n\") pod \"payload-pre-processing-59fbc47bf8-4r6np\" (UID: \"e3ba54fc-16ee-4eee-9826-5cd838046f10\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np"
Sep 04 23:45:34.898411 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:34.898375    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np"
Sep 04 23:45:35.166625 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:35.166598    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np"]
Sep 04 23:45:35.170554 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:45:35.170524    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3ba54fc_16ee_4eee_9826_5cd838046f10.slice/crio-f21f2674005a9aa3876a0887ad85d1ee44ae3b0de5843e3e8ec1a249465bc02e WatchSource:0}: Error finding container f21f2674005a9aa3876a0887ad85d1ee44ae3b0de5843e3e8ec1a249465bc02e: Status 404 returned error can't find the container with id f21f2674005a9aa3876a0887ad85d1ee44ae3b0de5843e3e8ec1a249465bc02e
Sep 04 23:45:35.478104 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:35.478076    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np" event={"ID":"e3ba54fc-16ee-4eee-9826-5cd838046f10","Type":"ContainerStarted","Data":"f21f2674005a9aa3876a0887ad85d1ee44ae3b0de5843e3e8ec1a249465bc02e"}
Sep 04 23:45:37.489976 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:37.489938    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np" event={"ID":"e3ba54fc-16ee-4eee-9826-5cd838046f10","Type":"ContainerStarted","Data":"b021158f367db4641901ce7c384c20110b50020453e4fc0956eada614571f815"}
Sep 04 23:45:37.490356 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:37.490039    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np"
Sep 04 23:45:37.506213 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:37.506163    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np" podStartSLOduration=1.595383883 podStartE2EDuration="3.506142995s" totalImagesPullingTime="1.910759112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:45:37.504303758 +0000 UTC m=+525.752304291" watchObservedRunningTime="2026-09-04 23:45:37.506142995 +0000 UTC m=+525.754143515"
Sep 04 23:45:46.056955 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:46.056852    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:46.157111 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:46.157078    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:48.497466 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:48.497433    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4r6np"
Sep 04 23:45:54.163399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:54.163363    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:59.217116 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.217078    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"]
Sep 04 23:45:59.220649 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.220626    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.223198 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.223178    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 04 23:45:59.224406 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.224387    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 04 23:45:59.224510 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.224422    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rfvrs\""
Sep 04 23:45:59.224510 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.224426    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 04 23:45:59.230208 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.230190    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"]
Sep 04 23:45:59.240328 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.240297    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:45:59.258068 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.258045    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9bab180-6f5b-454e-9662-0c45bdd82baf-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.258161 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.258090    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9bab180-6f5b-454e-9662-0c45bdd82baf-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.258161 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.258106    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9bab180-6f5b-454e-9662-0c45bdd82baf-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.258161 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.258125    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9bab180-6f5b-454e-9662-0c45bdd82baf-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.258269 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.258191    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxpmh\" (UniqueName: \"kubernetes.io/projected/c9bab180-6f5b-454e-9662-0c45bdd82baf-kube-api-access-hxpmh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.258303 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.258267    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9bab180-6f5b-454e-9662-0c45bdd82baf-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.358799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.358773    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9bab180-6f5b-454e-9662-0c45bdd82baf-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.358929 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.358817    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9bab180-6f5b-454e-9662-0c45bdd82baf-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.358929 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.358832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9bab180-6f5b-454e-9662-0c45bdd82baf-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.359031 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.358965    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9bab180-6f5b-454e-9662-0c45bdd82baf-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.359031 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.359019    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxpmh\" (UniqueName: \"kubernetes.io/projected/c9bab180-6f5b-454e-9662-0c45bdd82baf-kube-api-access-hxpmh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.359129 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.359091    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9bab180-6f5b-454e-9662-0c45bdd82baf-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.359209 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.359186    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9bab180-6f5b-454e-9662-0c45bdd82baf-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.359339 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.359256    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9bab180-6f5b-454e-9662-0c45bdd82baf-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.359414 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.359397    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9bab180-6f5b-454e-9662-0c45bdd82baf-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.361328 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.361296    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9bab180-6f5b-454e-9662-0c45bdd82baf-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.361589 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.361570    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9bab180-6f5b-454e-9662-0c45bdd82baf-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.367185 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.367159    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxpmh\" (UniqueName: \"kubernetes.io/projected/c9bab180-6f5b-454e-9662-0c45bdd82baf-kube-api-access-hxpmh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gfqvz\" (UID: \"c9bab180-6f5b-454e-9662-0c45bdd82baf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.531256 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.531192    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:45:59.801588 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:45:59.801562    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"]
Sep 04 23:45:59.803032 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:45:59.803008    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9bab180_6f5b_454e_9662_0c45bdd82baf.slice/crio-e43830a2db3cb2965aa794104ddb48a74d0dffeda984f1fb7cef226882f43772 WatchSource:0}: Error finding container e43830a2db3cb2965aa794104ddb48a74d0dffeda984f1fb7cef226882f43772: Status 404 returned error can't find the container with id e43830a2db3cb2965aa794104ddb48a74d0dffeda984f1fb7cef226882f43772
Sep 04 23:46:00.596873 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:00.596823    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz" event={"ID":"c9bab180-6f5b-454e-9662-0c45bdd82baf","Type":"ContainerStarted","Data":"e43830a2db3cb2965aa794104ddb48a74d0dffeda984f1fb7cef226882f43772"}
Sep 04 23:46:01.602524 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:01.602489    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz" event={"ID":"c9bab180-6f5b-454e-9662-0c45bdd82baf","Type":"ContainerStarted","Data":"8fab034f948a740fd3a6ae417a06fd46d42fbef5e29bb8c2a3aada7035e96ba3"}
Sep 04 23:46:01.618957 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:01.618885    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz" podStartSLOduration=1.505239948 podStartE2EDuration="2.618870724s" totalImagesPullingTime="1.113630776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:45:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:46:01.617363943 +0000 UTC m=+549.865364490" watchObservedRunningTime="2026-09-04 23:46:01.618870724 +0000 UTC m=+549.866871285"
Sep 04 23:46:06.991050 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:06.991015    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:46:07.040633 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:07.040596    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:46:09.532128 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:09.532093    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:46:09.532611 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:09.532156    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:46:09.544659 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:09.544634    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:46:09.648102 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:09.648081    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gfqvz"
Sep 04 23:46:14.848846 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:14.848809    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:46:22.213603 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.213573    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"]
Sep 04 23:46:22.217045 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.217021    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.219528 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.219510    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 04 23:46:22.225171 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.225146    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"]
Sep 04 23:46:22.239333 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.239310    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:46:22.331567 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.331542    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d911a1b7-e4b8-4984-9521-30310391bc95-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.331666 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.331572    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d911a1b7-e4b8-4984-9521-30310391bc95-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.331666 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.331589    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d911a1b7-e4b8-4984-9521-30310391bc95-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.331666 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.331605    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d911a1b7-e4b8-4984-9521-30310391bc95-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.331776 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.331675    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-454pk\" (UniqueName: \"kubernetes.io/projected/d911a1b7-e4b8-4984-9521-30310391bc95-kube-api-access-454pk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.331776 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.331715    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d911a1b7-e4b8-4984-9521-30310391bc95-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.432562 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.432536    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d911a1b7-e4b8-4984-9521-30310391bc95-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.432669 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.432568    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d911a1b7-e4b8-4984-9521-30310391bc95-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.432669 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.432583    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d911a1b7-e4b8-4984-9521-30310391bc95-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.432669 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.432601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d911a1b7-e4b8-4984-9521-30310391bc95-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.432669 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.432641    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-454pk\" (UniqueName: \"kubernetes.io/projected/d911a1b7-e4b8-4984-9521-30310391bc95-kube-api-access-454pk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.432860 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.432839    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d911a1b7-e4b8-4984-9521-30310391bc95-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.432962 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.432940    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d911a1b7-e4b8-4984-9521-30310391bc95-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.433021 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.433002    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d911a1b7-e4b8-4984-9521-30310391bc95-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.433107 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.433084    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d911a1b7-e4b8-4984-9521-30310391bc95-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.434982 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.434953    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d911a1b7-e4b8-4984-9521-30310391bc95-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.435272 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.435253    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d911a1b7-e4b8-4984-9521-30310391bc95-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.440421 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.440404    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-454pk\" (UniqueName: \"kubernetes.io/projected/d911a1b7-e4b8-4984-9521-30310391bc95-kube-api-access-454pk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-csgh4\" (UID: \"d911a1b7-e4b8-4984-9521-30310391bc95\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.528236 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.528182    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:22.798090 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:22.798056    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"]
Sep 04 23:46:22.800730 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:46:22.800703    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd911a1b7_e4b8_4984_9521_30310391bc95.slice/crio-f23c5f445f34e54e4a401104abeacbbd971868654ce0a5b085ac0e1f758c7501 WatchSource:0}: Error finding container f23c5f445f34e54e4a401104abeacbbd971868654ce0a5b085ac0e1f758c7501: Status 404 returned error can't find the container with id f23c5f445f34e54e4a401104abeacbbd971868654ce0a5b085ac0e1f758c7501
Sep 04 23:46:23.714556 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:23.714515    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4" event={"ID":"d911a1b7-e4b8-4984-9521-30310391bc95","Type":"ContainerStarted","Data":"1066acc13e8624df1ef12f9633158d753a49d0d88121e521b6e74a5ae511fc2d"}
Sep 04 23:46:23.714949 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:23.714608    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4" event={"ID":"d911a1b7-e4b8-4984-9521-30310391bc95","Type":"ContainerStarted","Data":"f23c5f445f34e54e4a401104abeacbbd971868654ce0a5b085ac0e1f758c7501"}
Sep 04 23:46:23.731558 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:23.731510    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4" podStartSLOduration=1.550045489 podStartE2EDuration="1.731496053s" totalImagesPullingTime="181.450564ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:46:23.729968146 +0000 UTC m=+571.977968692" watchObservedRunningTime="2026-09-04 23:46:23.731496053 +0000 UTC m=+571.979496570"
Sep 04 23:46:32.528678 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:32.528643    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:32.529131 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:32.528710    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:32.540947 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:32.540923    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:32.763720 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:32.763694    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-csgh4"
Sep 04 23:46:37.119387 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.119347    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"]
Sep 04 23:46:37.124649 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.124626    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.127242 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.127223    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 04 23:46:37.131313 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.131289    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"]
Sep 04 23:46:37.249000 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.248971    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.249142 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.249005    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.249142 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.249084    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.249142 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.249103    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.249142 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.249119    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.249142 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.249136    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmcm4\" (UniqueName: \"kubernetes.io/projected/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-kube-api-access-dmcm4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.310180 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.310149    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:46:37.349546 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.349516    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.349660 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.349557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.349660 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.349581    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.349660 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.349609    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmcm4\" (UniqueName: \"kubernetes.io/projected/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-kube-api-access-dmcm4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.349660 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.349655    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.349839 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.349684    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.349961 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.349938    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.350027 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.350005    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.350188 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.350165    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.351927 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.351891    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.352212 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.352194    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.357501 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.357483    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmcm4\" (UniqueName: \"kubernetes.io/projected/8cd22c48-6a0a-442e-bb85-c05e0a4881d8-kube-api-access-dmcm4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf\" (UID: \"8cd22c48-6a0a-442e-bb85-c05e0a4881d8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.435772 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.435705    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:37.707710 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.707687    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"]
Sep 04 23:46:37.710270 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:46:37.710243    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cd22c48_6a0a_442e_bb85_c05e0a4881d8.slice/crio-4bcbed009e22466344ecd4a3eaecbb1422697224df0b55252899a22321949e7c WatchSource:0}: Error finding container 4bcbed009e22466344ecd4a3eaecbb1422697224df0b55252899a22321949e7c: Status 404 returned error can't find the container with id 4bcbed009e22466344ecd4a3eaecbb1422697224df0b55252899a22321949e7c
Sep 04 23:46:37.775620 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:37.775588    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf" event={"ID":"8cd22c48-6a0a-442e-bb85-c05e0a4881d8","Type":"ContainerStarted","Data":"4bcbed009e22466344ecd4a3eaecbb1422697224df0b55252899a22321949e7c"}
Sep 04 23:46:38.780829 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:38.780787    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf" event={"ID":"8cd22c48-6a0a-442e-bb85-c05e0a4881d8","Type":"ContainerStarted","Data":"c4fb1ecf3a0bca3e528ce7d2770d4f3b44737cc7c349b08d66ab148bfcc13aed"}
Sep 04 23:46:38.798438 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:38.798384    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf" podStartSLOduration=1.633116035 podStartE2EDuration="1.798368324s" totalImagesPullingTime="165.252289ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:46:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:46:38.795811951 +0000 UTC m=+587.043812484" watchObservedRunningTime="2026-09-04 23:46:38.798368324 +0000 UTC m=+587.046368839"
Sep 04 23:46:42.406640 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:42.406606    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:46:43.720033 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:43.719999    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:46:46.689261 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:46.689223    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:46:46.789233 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:46.789206    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:46:47.436567 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:47.436529    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:47.436750 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:47.436591    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:47.449375 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:47.449350    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:47.835237 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:47.835163    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf"
Sep 04 23:46:52.351071 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:46:52.351026    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:47:35.536368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:35.536338    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:47:57.818977 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:57.818943    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69b4b9b8d-sdgv5"]
Sep 04 23:47:57.822340 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:57.822321    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69b4b9b8d-sdgv5"
Sep 04 23:47:57.828666 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:57.828641    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69b4b9b8d-sdgv5"]
Sep 04 23:47:57.929781 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:57.929741    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2283e3ad-02f4-470e-9e4d-b8cfd4912968-tls-cert\") pod \"authorino-69b4b9b8d-sdgv5\" (UID: \"2283e3ad-02f4-470e-9e4d-b8cfd4912968\") " pod="kuadrant-system/authorino-69b4b9b8d-sdgv5"
Sep 04 23:47:57.929978 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:57.929794    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf5j8\" (UniqueName: \"kubernetes.io/projected/2283e3ad-02f4-470e-9e4d-b8cfd4912968-kube-api-access-sf5j8\") pod \"authorino-69b4b9b8d-sdgv5\" (UID: \"2283e3ad-02f4-470e-9e4d-b8cfd4912968\") " pod="kuadrant-system/authorino-69b4b9b8d-sdgv5"
Sep 04 23:47:58.031100 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:58.031060    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2283e3ad-02f4-470e-9e4d-b8cfd4912968-tls-cert\") pod \"authorino-69b4b9b8d-sdgv5\" (UID: \"2283e3ad-02f4-470e-9e4d-b8cfd4912968\") " pod="kuadrant-system/authorino-69b4b9b8d-sdgv5"
Sep 04 23:47:58.031268 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:58.031132    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf5j8\" (UniqueName: \"kubernetes.io/projected/2283e3ad-02f4-470e-9e4d-b8cfd4912968-kube-api-access-sf5j8\") pod \"authorino-69b4b9b8d-sdgv5\" (UID: \"2283e3ad-02f4-470e-9e4d-b8cfd4912968\") " pod="kuadrant-system/authorino-69b4b9b8d-sdgv5"
Sep 04 23:47:58.033647 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:58.033624    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2283e3ad-02f4-470e-9e4d-b8cfd4912968-tls-cert\") pod \"authorino-69b4b9b8d-sdgv5\" (UID: \"2283e3ad-02f4-470e-9e4d-b8cfd4912968\") " pod="kuadrant-system/authorino-69b4b9b8d-sdgv5"
Sep 04 23:47:58.038979 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:58.038959    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf5j8\" (UniqueName: \"kubernetes.io/projected/2283e3ad-02f4-470e-9e4d-b8cfd4912968-kube-api-access-sf5j8\") pod \"authorino-69b4b9b8d-sdgv5\" (UID: \"2283e3ad-02f4-470e-9e4d-b8cfd4912968\") " pod="kuadrant-system/authorino-69b4b9b8d-sdgv5"
Sep 04 23:47:58.132299 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:58.132274    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69b4b9b8d-sdgv5"
Sep 04 23:47:58.400291 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:58.400268    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69b4b9b8d-sdgv5"]
Sep 04 23:47:58.402888 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:47:58.402859    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2283e3ad_02f4_470e_9e4d_b8cfd4912968.slice/crio-3d1f9048ac4d24995745ddca318b5364760efb25cf68340adefec814ea50dc66 WatchSource:0}: Error finding container 3d1f9048ac4d24995745ddca318b5364760efb25cf68340adefec814ea50dc66: Status 404 returned error can't find the container with id 3d1f9048ac4d24995745ddca318b5364760efb25cf68340adefec814ea50dc66
Sep 04 23:47:58.404322 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:58.404306    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 23:47:59.143826 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:47:59.143795    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69b4b9b8d-sdgv5" event={"ID":"2283e3ad-02f4-470e-9e4d-b8cfd4912968","Type":"ContainerStarted","Data":"3d1f9048ac4d24995745ddca318b5364760efb25cf68340adefec814ea50dc66"}
Sep 04 23:48:00.150472 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:00.150436    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69b4b9b8d-sdgv5" event={"ID":"2283e3ad-02f4-470e-9e4d-b8cfd4912968","Type":"ContainerStarted","Data":"27ac5b5d73263677cd35166a300c2be60a067c1497f8dfdc6459afed46149ebb"}
Sep 04 23:48:00.166960 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:00.166885    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-69b4b9b8d-sdgv5" podStartSLOduration=2.143331519 podStartE2EDuration="3.166863462s" totalImagesPullingTime="1.023531943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:47:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:48:00.166583549 +0000 UTC m=+668.414584069" watchObservedRunningTime="2026-09-04 23:48:00.166863462 +0000 UTC m=+668.414863979"
Sep 04 23:48:00.205053 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:00.205021    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f9d9dd6c7-lt888"]
Sep 04 23:48:00.205519 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:00.205488    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6f9d9dd6c7-lt888" podUID="2e59d932-7594-443d-8058-00f1f6673968" containerName="authorino" containerID="cri-o://c7a97367667c89169fcb63b7b76260b014bbfa7c6ee042c638d9a830b71302a8" gracePeriod=30
Sep 04 23:48:00.458259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:00.458236    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f9d9dd6c7-lt888"
Sep 04 23:48:00.554221 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:00.554189    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvjkx\" (UniqueName: \"kubernetes.io/projected/2e59d932-7594-443d-8058-00f1f6673968-kube-api-access-tvjkx\") pod \"2e59d932-7594-443d-8058-00f1f6673968\" (UID: \"2e59d932-7594-443d-8058-00f1f6673968\") "
Sep 04 23:48:00.554425 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:00.554237    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e59d932-7594-443d-8058-00f1f6673968-tls-cert\") pod \"2e59d932-7594-443d-8058-00f1f6673968\" (UID: \"2e59d932-7594-443d-8058-00f1f6673968\") "
Sep 04 23:48:00.556453 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:00.556425    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e59d932-7594-443d-8058-00f1f6673968-kube-api-access-tvjkx" (OuterVolumeSpecName: "kube-api-access-tvjkx") pod "2e59d932-7594-443d-8058-00f1f6673968" (UID: "2e59d932-7594-443d-8058-00f1f6673968"). InnerVolumeSpecName "kube-api-access-tvjkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:48:00.564669 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:00.564563    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e59d932-7594-443d-8058-00f1f6673968-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2e59d932-7594-443d-8058-00f1f6673968" (UID: "2e59d932-7594-443d-8058-00f1f6673968"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:48:00.655270 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:00.655230    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvjkx\" (UniqueName: \"kubernetes.io/projected/2e59d932-7594-443d-8058-00f1f6673968-kube-api-access-tvjkx\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:48:00.655270 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:00.655260    2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2e59d932-7594-443d-8058-00f1f6673968-tls-cert\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:48:00.689960 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:00.689846    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:00.789790 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:00.789757    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:01.157697 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:01.157665    2564 generic.go:356] "Generic (PLEG): container finished" podID="2e59d932-7594-443d-8058-00f1f6673968" containerID="c7a97367667c89169fcb63b7b76260b014bbfa7c6ee042c638d9a830b71302a8" exitCode=0
Sep 04 23:48:01.158288 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:01.157996    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f9d9dd6c7-lt888" event={"ID":"2e59d932-7594-443d-8058-00f1f6673968","Type":"ContainerDied","Data":"c7a97367667c89169fcb63b7b76260b014bbfa7c6ee042c638d9a830b71302a8"}
Sep 04 23:48:01.158400 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:01.158322    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f9d9dd6c7-lt888" event={"ID":"2e59d932-7594-443d-8058-00f1f6673968","Type":"ContainerDied","Data":"fda40ac6f6709ac5218a3b6b941618c64ca28198f20da78339f135ef270b10ec"}
Sep 04 23:48:01.158400 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:01.158353    2564 scope.go:117] "RemoveContainer" containerID="c7a97367667c89169fcb63b7b76260b014bbfa7c6ee042c638d9a830b71302a8"
Sep 04 23:48:01.158400 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:01.158025    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f9d9dd6c7-lt888"
Sep 04 23:48:01.175558 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:01.175538    2564 scope.go:117] "RemoveContainer" containerID="c7a97367667c89169fcb63b7b76260b014bbfa7c6ee042c638d9a830b71302a8"
Sep 04 23:48:01.177371 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:48:01.177345    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7a97367667c89169fcb63b7b76260b014bbfa7c6ee042c638d9a830b71302a8\": container with ID starting with c7a97367667c89169fcb63b7b76260b014bbfa7c6ee042c638d9a830b71302a8 not found: ID does not exist" containerID="c7a97367667c89169fcb63b7b76260b014bbfa7c6ee042c638d9a830b71302a8"
Sep 04 23:48:01.177513 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:01.177488    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7a97367667c89169fcb63b7b76260b014bbfa7c6ee042c638d9a830b71302a8"} err="failed to get container status \"c7a97367667c89169fcb63b7b76260b014bbfa7c6ee042c638d9a830b71302a8\": rpc error: code = NotFound desc = could not find container \"c7a97367667c89169fcb63b7b76260b014bbfa7c6ee042c638d9a830b71302a8\": container with ID starting with c7a97367667c89169fcb63b7b76260b014bbfa7c6ee042c638d9a830b71302a8 not found: ID does not exist"
Sep 04 23:48:01.191976 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:01.191948    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f9d9dd6c7-lt888"]
Sep 04 23:48:01.196299 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:01.196274    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f9d9dd6c7-lt888"]
Sep 04 23:48:01.775298 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:01.775261    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:01.891869 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:01.891832    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:02.176210 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:02.176183    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e59d932-7594-443d-8058-00f1f6673968" path="/var/lib/kubelet/pods/2e59d932-7594-443d-8058-00f1f6673968/volumes"
Sep 04 23:48:16.850401 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:16.850359    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:16.964427 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:16.964391    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:17.293444 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:17.293409    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:17.539349 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:17.539319    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:18.286159 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:18.286122    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:18.386679 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:18.386640    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:18.633153 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:18.633120    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:18.819543 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:18.819501    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7642-4df26a"]
Sep 04 23:48:18.819944 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:18.819926    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e59d932-7594-443d-8058-00f1f6673968" containerName="authorino"
Sep 04 23:48:18.820035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:18.819948    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e59d932-7594-443d-8058-00f1f6673968" containerName="authorino"
Sep 04 23:48:18.820090 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:18.820061    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e59d932-7594-443d-8058-00f1f6673968" containerName="authorino"
Sep 04 23:48:18.822974 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:18.822954    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7642-4df26a"
Sep 04 23:48:18.825886 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:18.825863    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5vfk6\""
Sep 04 23:48:18.827515 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:18.827496    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7642-4df26a"]
Sep 04 23:48:18.919637 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:18.919564    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nglk8\" (UniqueName: \"kubernetes.io/projected/bd49a8ff-506b-4fa4-8015-51334291a657-kube-api-access-nglk8\") pod \"test-curl-7642-4df26a\" (UID: \"bd49a8ff-506b-4fa4-8015-51334291a657\") " pod="opendatahub/test-curl-7642-4df26a"
Sep 04 23:48:19.020571 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:19.020536    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nglk8\" (UniqueName: \"kubernetes.io/projected/bd49a8ff-506b-4fa4-8015-51334291a657-kube-api-access-nglk8\") pod \"test-curl-7642-4df26a\" (UID: \"bd49a8ff-506b-4fa4-8015-51334291a657\") " pod="opendatahub/test-curl-7642-4df26a"
Sep 04 23:48:19.031552 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:19.031523    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nglk8\" (UniqueName: \"kubernetes.io/projected/bd49a8ff-506b-4fa4-8015-51334291a657-kube-api-access-nglk8\") pod \"test-curl-7642-4df26a\" (UID: \"bd49a8ff-506b-4fa4-8015-51334291a657\") " pod="opendatahub/test-curl-7642-4df26a"
Sep 04 23:48:19.134822 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:19.134787    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7642-4df26a"
Sep 04 23:48:19.402965 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:19.402942    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7642-4df26a"]
Sep 04 23:48:19.405443 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:48:19.405416    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd49a8ff_506b_4fa4_8015_51334291a657.slice/crio-2ba864c1b2ba40e0f9f2edb9d27304db8cb8f42e04adab82b94da2f96b4a1d0a WatchSource:0}: Error finding container 2ba864c1b2ba40e0f9f2edb9d27304db8cb8f42e04adab82b94da2f96b4a1d0a: Status 404 returned error can't find the container with id 2ba864c1b2ba40e0f9f2edb9d27304db8cb8f42e04adab82b94da2f96b4a1d0a
Sep 04 23:48:20.260759 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:20.260728    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7642-4df26a" event={"ID":"bd49a8ff-506b-4fa4-8015-51334291a657","Type":"ContainerStarted","Data":"2ba864c1b2ba40e0f9f2edb9d27304db8cb8f42e04adab82b94da2f96b4a1d0a"}
Sep 04 23:48:20.369775 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:20.369688    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:20.485377 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:20.485331    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:21.317924 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:21.317863    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:21.414584 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:21.414535    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:21.693260 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:21.693233    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:21.799310 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:21.799272    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:21.892699 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:21.892668    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:21.893466 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:21.893443    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"]
Sep 04 23:48:21.896869 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:21.896850    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"
Sep 04 23:48:21.899484 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:21.899464    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-d8b443\""
Sep 04 23:48:21.899684 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:21.899665    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-d8b443-dockercfg-sth8z\""
Sep 04 23:48:21.907486 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:21.907462    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"]
Sep 04 23:48:21.925745 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:21.925679    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"]
Sep 04 23:48:21.928952 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:21.928934    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"
Sep 04 23:48:21.939491 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:21.939472    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"]
Sep 04 23:48:22.047281 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.047243    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr6m7\" (UniqueName: \"kubernetes.io/projected/6c87c053-a412-48f7-9981-849412a38263-kube-api-access-jr6m7\") pod \"payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc\" (UID: \"6c87c053-a412-48f7-9981-849412a38263\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"
Sep 04 23:48:22.047446 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.047321    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c87c053-a412-48f7-9981-849412a38263-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc\" (UID: \"6c87c053-a412-48f7-9981-849412a38263\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"
Sep 04 23:48:22.047446 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.047346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b8976c5-6336-4a19-8749-5d4d966178c9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf\" (UID: \"2b8976c5-6336-4a19-8749-5d4d966178c9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"
Sep 04 23:48:22.047446 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.047369    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88sv9\" (UniqueName: \"kubernetes.io/projected/2b8976c5-6336-4a19-8749-5d4d966178c9-kube-api-access-88sv9\") pod \"payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf\" (UID: \"2b8976c5-6336-4a19-8749-5d4d966178c9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"
Sep 04 23:48:22.148721 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.148680    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c87c053-a412-48f7-9981-849412a38263-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc\" (UID: \"6c87c053-a412-48f7-9981-849412a38263\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"
Sep 04 23:48:22.148946 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.148733    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b8976c5-6336-4a19-8749-5d4d966178c9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf\" (UID: \"2b8976c5-6336-4a19-8749-5d4d966178c9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"
Sep 04 23:48:22.148946 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.148772    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88sv9\" (UniqueName: \"kubernetes.io/projected/2b8976c5-6336-4a19-8749-5d4d966178c9-kube-api-access-88sv9\") pod \"payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf\" (UID: \"2b8976c5-6336-4a19-8749-5d4d966178c9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"
Sep 04 23:48:22.148946 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.148852    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr6m7\" (UniqueName: \"kubernetes.io/projected/6c87c053-a412-48f7-9981-849412a38263-kube-api-access-jr6m7\") pod \"payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc\" (UID: \"6c87c053-a412-48f7-9981-849412a38263\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"
Sep 04 23:48:22.149706 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.149678    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b8976c5-6336-4a19-8749-5d4d966178c9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf\" (UID: \"2b8976c5-6336-4a19-8749-5d4d966178c9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"
Sep 04 23:48:22.149812 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.149708    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c87c053-a412-48f7-9981-849412a38263-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc\" (UID: \"6c87c053-a412-48f7-9981-849412a38263\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"
Sep 04 23:48:22.157992 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.157969    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88sv9\" (UniqueName: \"kubernetes.io/projected/2b8976c5-6336-4a19-8749-5d4d966178c9-kube-api-access-88sv9\") pod \"payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf\" (UID: \"2b8976c5-6336-4a19-8749-5d4d966178c9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"
Sep 04 23:48:22.158101 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.158038    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr6m7\" (UniqueName: \"kubernetes.io/projected/6c87c053-a412-48f7-9981-849412a38263-kube-api-access-jr6m7\") pod \"payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc\" (UID: \"6c87c053-a412-48f7-9981-849412a38263\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"
Sep 04 23:48:22.207284 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.207218    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"
Sep 04 23:48:22.239155 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.239123    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"
Sep 04 23:48:22.279658 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.278119    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7642-4df26a" event={"ID":"bd49a8ff-506b-4fa4-8015-51334291a657","Type":"ContainerStarted","Data":"c1d44d23f2e1968efe9227df7cc2113f7de47957ec410d5aaff9d9b8cdb53866"}
Sep 04 23:48:22.293776 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.293721    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7642-4df26a" podStartSLOduration=2.030483663 podStartE2EDuration="4.293704326s" totalImagesPullingTime="2.263220663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:48:22.292368324 +0000 UTC m=+690.540368841" watchObservedRunningTime="2026-09-04 23:48:22.293704326 +0000 UTC m=+690.541704841"
Sep 04 23:48:22.484116 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.484092    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"]
Sep 04 23:48:22.485816 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:48:22.485789    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c87c053_a412_48f7_9981_849412a38263.slice/crio-766411719d87de8430bc9c06543a38198348dbcf691aba0da11aa5df0b6bbb23 WatchSource:0}: Error finding container 766411719d87de8430bc9c06543a38198348dbcf691aba0da11aa5df0b6bbb23: Status 404 returned error can't find the container with id 766411719d87de8430bc9c06543a38198348dbcf691aba0da11aa5df0b6bbb23
Sep 04 23:48:22.522300 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.522279    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"]
Sep 04 23:48:22.524074 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:48:22.524054    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b8976c5_6336_4a19_8749_5d4d966178c9.slice/crio-8046700316a1d71d8d2f236a711c03274dc1798bd502c46f1fc6860f55f95c9e WatchSource:0}: Error finding container 8046700316a1d71d8d2f236a711c03274dc1798bd502c46f1fc6860f55f95c9e: Status 404 returned error can't find the container with id 8046700316a1d71d8d2f236a711c03274dc1798bd502c46f1fc6860f55f95c9e
Sep 04 23:48:22.842651 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.842560    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7642-4df26a"]
Sep 04 23:48:22.845000 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:22.844969    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7642-4df26a"]
Sep 04 23:48:23.062969 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.062936    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7642-2d5d01"]
Sep 04 23:48:23.063474 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.063457    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd49a8ff-506b-4fa4-8015-51334291a657" containerName="test-curl-7642-4df26a"
Sep 04 23:48:23.063559 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.063476    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd49a8ff-506b-4fa4-8015-51334291a657" containerName="test-curl-7642-4df26a"
Sep 04 23:48:23.063616 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.063574    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd49a8ff-506b-4fa4-8015-51334291a657" containerName="test-curl-7642-4df26a"
Sep 04 23:48:23.066921 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.066873    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7642-2d5d01"
Sep 04 23:48:23.069882 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.069855    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7642-2d5d01"]
Sep 04 23:48:23.157876 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.157851    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrzjt\" (UniqueName: \"kubernetes.io/projected/e4f71f3c-823d-4d8d-b200-dfa2f1c1c728-kube-api-access-nrzjt\") pod \"test-curl-7642-2d5d01\" (UID: \"e4f71f3c-823d-4d8d-b200-dfa2f1c1c728\") " pod="opendatahub/test-curl-7642-2d5d01"
Sep 04 23:48:23.258810 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.258784    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrzjt\" (UniqueName: \"kubernetes.io/projected/e4f71f3c-823d-4d8d-b200-dfa2f1c1c728-kube-api-access-nrzjt\") pod \"test-curl-7642-2d5d01\" (UID: \"e4f71f3c-823d-4d8d-b200-dfa2f1c1c728\") " pod="opendatahub/test-curl-7642-2d5d01"
Sep 04 23:48:23.266991 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.266964    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrzjt\" (UniqueName: \"kubernetes.io/projected/e4f71f3c-823d-4d8d-b200-dfa2f1c1c728-kube-api-access-nrzjt\") pod \"test-curl-7642-2d5d01\" (UID: \"e4f71f3c-823d-4d8d-b200-dfa2f1c1c728\") " pod="opendatahub/test-curl-7642-2d5d01"
Sep 04 23:48:23.284843 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.284818    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc" event={"ID":"6c87c053-a412-48f7-9981-849412a38263","Type":"ContainerStarted","Data":"a8655dece554dcd20966d640d96732a88f14bacbd43fbb0b39d13108c27905d5"}
Sep 04 23:48:23.285159 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.284871    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc" event={"ID":"6c87c053-a412-48f7-9981-849412a38263","Type":"ContainerStarted","Data":"766411719d87de8430bc9c06543a38198348dbcf691aba0da11aa5df0b6bbb23"}
Sep 04 23:48:23.285159 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.284927    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"
Sep 04 23:48:23.286841 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.286811    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf" event={"ID":"2b8976c5-6336-4a19-8749-5d4d966178c9","Type":"ContainerStarted","Data":"3f247f6335c0bf3cbf7d75efcefee001a37808bf58e17686ce73df7929908fbb"}
Sep 04 23:48:23.286986 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.286866    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf" event={"ID":"2b8976c5-6336-4a19-8749-5d4d966178c9","Type":"ContainerStarted","Data":"8046700316a1d71d8d2f236a711c03274dc1798bd502c46f1fc6860f55f95c9e"}
Sep 04 23:48:23.287063 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.287022    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"
Sep 04 23:48:23.287130 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:48:23.287094    2564 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7642-4df26a" container="test-curl-7642-4df26a"
Sep 04 23:48:23.303685 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.303644    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc" podStartSLOduration=2.303626559 podStartE2EDuration="2.303626559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:48:23.300242403 +0000 UTC m=+691.548242922" watchObservedRunningTime="2026-09-04 23:48:23.303626559 +0000 UTC m=+691.551627076"
Sep 04 23:48:23.316394 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.316355    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf" podStartSLOduration=2.316342152 podStartE2EDuration="2.316342152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:48:23.315301922 +0000 UTC m=+691.563302458" watchObservedRunningTime="2026-09-04 23:48:23.316342152 +0000 UTC m=+691.564342667"
Sep 04 23:48:23.380538 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.380514    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7642-2d5d01"
Sep 04 23:48:23.649231 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:23.649202    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7642-2d5d01"]
Sep 04 23:48:23.650832 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:48:23.650800    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4f71f3c_823d_4d8d_b200_dfa2f1c1c728.slice/crio-52353d6caec549610359ada9479b461c1cf120966b60477982b9af63a907af3e WatchSource:0}: Error finding container 52353d6caec549610359ada9479b461c1cf120966b60477982b9af63a907af3e: Status 404 returned error can't find the container with id 52353d6caec549610359ada9479b461c1cf120966b60477982b9af63a907af3e
Sep 04 23:48:24.294480 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:24.294441    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7642-2d5d01" event={"ID":"e4f71f3c-823d-4d8d-b200-dfa2f1c1c728","Type":"ContainerStarted","Data":"d7628bab055d3c4e8e1445a136b7953e85e5c66116da2747dd59b28b2cfc7254"}
Sep 04 23:48:24.294999 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:24.294493    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7642-2d5d01" event={"ID":"e4f71f3c-823d-4d8d-b200-dfa2f1c1c728","Type":"ContainerStarted","Data":"52353d6caec549610359ada9479b461c1cf120966b60477982b9af63a907af3e"}
Sep 04 23:48:24.294999 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:24.294572    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7642-4df26a" podUID="bd49a8ff-506b-4fa4-8015-51334291a657" containerName="test-curl-7642-4df26a" containerID="cri-o://c1d44d23f2e1968efe9227df7cc2113f7de47957ec410d5aaff9d9b8cdb53866" gracePeriod=2
Sep 04 23:48:24.307986 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:24.307947    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7642-2d5d01" podStartSLOduration=1.307933534 podStartE2EDuration="1.307933534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:48:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:48:24.30742926 +0000 UTC m=+692.555429776" watchObservedRunningTime="2026-09-04 23:48:24.307933534 +0000 UTC m=+692.555934050"
Sep 04 23:48:24.321662 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:24.321640    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:24.381567 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:24.381530    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:24.590878 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:24.590796    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:24.839879 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:24.839839    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:24.915786 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:24.915751    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7642-2d5d01"]
Sep 04 23:48:24.919613 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:24.919586    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7642-2d5d01"]
Sep 04 23:48:25.125272 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:25.125237    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:25.299758 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:48:25.299679    2564 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7642-2d5d01" container="test-curl-7642-2d5d01"
Sep 04 23:48:25.413731 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:25.413700    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:25.690086 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:25.690052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:25.810561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:25.810529    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:26.305158 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:26.305124    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7642-2d5d01" podUID="e4f71f3c-823d-4d8d-b200-dfa2f1c1c728" containerName="test-curl-7642-2d5d01" containerID="cri-o://d7628bab055d3c4e8e1445a136b7953e85e5c66116da2747dd59b28b2cfc7254" gracePeriod=2
Sep 04 23:48:26.466256 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:26.466234    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7642-4df26a"
Sep 04 23:48:26.591496 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:26.591438    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nglk8\" (UniqueName: \"kubernetes.io/projected/bd49a8ff-506b-4fa4-8015-51334291a657-kube-api-access-nglk8\") pod \"bd49a8ff-506b-4fa4-8015-51334291a657\" (UID: \"bd49a8ff-506b-4fa4-8015-51334291a657\") "
Sep 04 23:48:26.593608 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:26.593591    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd49a8ff-506b-4fa4-8015-51334291a657-kube-api-access-nglk8" (OuterVolumeSpecName: "kube-api-access-nglk8") pod "bd49a8ff-506b-4fa4-8015-51334291a657" (UID: "bd49a8ff-506b-4fa4-8015-51334291a657"). InnerVolumeSpecName "kube-api-access-nglk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:48:26.692795 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:26.692772    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nglk8\" (UniqueName: \"kubernetes.io/projected/bd49a8ff-506b-4fa4-8015-51334291a657-kube-api-access-nglk8\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:48:27.311279 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:27.311248    2564 generic.go:356] "Generic (PLEG): container finished" podID="bd49a8ff-506b-4fa4-8015-51334291a657" containerID="c1d44d23f2e1968efe9227df7cc2113f7de47957ec410d5aaff9d9b8cdb53866" exitCode=137
Sep 04 23:48:27.311710 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:27.311306    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7642-4df26a"
Sep 04 23:48:27.311710 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:27.311313    2564 scope.go:117] "RemoveContainer" containerID="c1d44d23f2e1968efe9227df7cc2113f7de47957ec410d5aaff9d9b8cdb53866"
Sep 04 23:48:27.320798 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:27.320772    2564 scope.go:117] "RemoveContainer" containerID="c1d44d23f2e1968efe9227df7cc2113f7de47957ec410d5aaff9d9b8cdb53866"
Sep 04 23:48:27.321217 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:48:27.321192    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1d44d23f2e1968efe9227df7cc2113f7de47957ec410d5aaff9d9b8cdb53866\": container with ID starting with c1d44d23f2e1968efe9227df7cc2113f7de47957ec410d5aaff9d9b8cdb53866 not found: ID does not exist" containerID="c1d44d23f2e1968efe9227df7cc2113f7de47957ec410d5aaff9d9b8cdb53866"
Sep 04 23:48:27.321293 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:27.321231    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d44d23f2e1968efe9227df7cc2113f7de47957ec410d5aaff9d9b8cdb53866"} err="failed to get container status \"c1d44d23f2e1968efe9227df7cc2113f7de47957ec410d5aaff9d9b8cdb53866\": rpc error: code = NotFound desc = could not find container \"c1d44d23f2e1968efe9227df7cc2113f7de47957ec410d5aaff9d9b8cdb53866\": container with ID starting with c1d44d23f2e1968efe9227df7cc2113f7de47957ec410d5aaff9d9b8cdb53866 not found: ID does not exist"
Sep 04 23:48:27.322341 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:48:27.322319    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7642-4df26a\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="bd49a8ff-506b-4fa4-8015-51334291a657" pod="opendatahub/test-curl-7642-4df26a"
Sep 04 23:48:27.915692 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:27.915659    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:28.006392 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:28.006363    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:28.174955 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:28.174856    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd49a8ff-506b-4fa4-8015-51334291a657" path="/var/lib/kubelet/pods/bd49a8ff-506b-4fa4-8015-51334291a657/volumes"
Sep 04 23:48:28.484768 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:28.484740    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7642-2d5d01"
Sep 04 23:48:28.610249 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:28.610214    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrzjt\" (UniqueName: \"kubernetes.io/projected/e4f71f3c-823d-4d8d-b200-dfa2f1c1c728-kube-api-access-nrzjt\") pod \"e4f71f3c-823d-4d8d-b200-dfa2f1c1c728\" (UID: \"e4f71f3c-823d-4d8d-b200-dfa2f1c1c728\") "
Sep 04 23:48:28.612392 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:28.612361    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4f71f3c-823d-4d8d-b200-dfa2f1c1c728-kube-api-access-nrzjt" (OuterVolumeSpecName: "kube-api-access-nrzjt") pod "e4f71f3c-823d-4d8d-b200-dfa2f1c1c728" (UID: "e4f71f3c-823d-4d8d-b200-dfa2f1c1c728"). InnerVolumeSpecName "kube-api-access-nrzjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:48:28.711220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:28.711143    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrzjt\" (UniqueName: \"kubernetes.io/projected/e4f71f3c-823d-4d8d-b200-dfa2f1c1c728-kube-api-access-nrzjt\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:48:28.922469 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:28.922437    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:29.010687 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.010606    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:29.118137 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.118102    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:29.323634 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.323546    2564 generic.go:356] "Generic (PLEG): container finished" podID="e4f71f3c-823d-4d8d-b200-dfa2f1c1c728" containerID="d7628bab055d3c4e8e1445a136b7953e85e5c66116da2747dd59b28b2cfc7254" exitCode=137
Sep 04 23:48:29.323634 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.323589    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7642-2d5d01"
Sep 04 23:48:29.323802 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.323652    2564 scope.go:117] "RemoveContainer" containerID="d7628bab055d3c4e8e1445a136b7953e85e5c66116da2747dd59b28b2cfc7254"
Sep 04 23:48:29.332613 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.332599    2564 scope.go:117] "RemoveContainer" containerID="d7628bab055d3c4e8e1445a136b7953e85e5c66116da2747dd59b28b2cfc7254"
Sep 04 23:48:29.332854 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:48:29.332838    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7628bab055d3c4e8e1445a136b7953e85e5c66116da2747dd59b28b2cfc7254\": container with ID starting with d7628bab055d3c4e8e1445a136b7953e85e5c66116da2747dd59b28b2cfc7254 not found: ID does not exist" containerID="d7628bab055d3c4e8e1445a136b7953e85e5c66116da2747dd59b28b2cfc7254"
Sep 04 23:48:29.332926 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.332863    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7628bab055d3c4e8e1445a136b7953e85e5c66116da2747dd59b28b2cfc7254"} err="failed to get container status \"d7628bab055d3c4e8e1445a136b7953e85e5c66116da2747dd59b28b2cfc7254\": rpc error: code = NotFound desc = could not find container \"d7628bab055d3c4e8e1445a136b7953e85e5c66116da2747dd59b28b2cfc7254\": container with ID starting with d7628bab055d3c4e8e1445a136b7953e85e5c66116da2747dd59b28b2cfc7254 not found: ID does not exist"
Sep 04 23:48:29.333676 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:48:29.333654    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7642-2d5d01\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="e4f71f3c-823d-4d8d-b200-dfa2f1c1c728" pod="opendatahub/test-curl-7642-2d5d01"
Sep 04 23:48:29.510489 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.510457    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:29.596212 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.596137    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"]
Sep 04 23:48:29.596523 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.596511    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4f71f3c-823d-4d8d-b200-dfa2f1c1c728" containerName="test-curl-7642-2d5d01"
Sep 04 23:48:29.596583 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.596524    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4f71f3c-823d-4d8d-b200-dfa2f1c1c728" containerName="test-curl-7642-2d5d01"
Sep 04 23:48:29.596624 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.596588    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4f71f3c-823d-4d8d-b200-dfa2f1c1c728" containerName="test-curl-7642-2d5d01"
Sep 04 23:48:29.598826 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.598802    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:29.600853 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.600838    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"
Sep 04 23:48:29.603167 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:48:29.603141    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7642-2d5d01\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="e4f71f3c-823d-4d8d-b200-dfa2f1c1c728" pod="opendatahub/test-curl-7642-2d5d01"
Sep 04 23:48:29.603264 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.603207    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-eef2d470\""
Sep 04 23:48:29.603417 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.603403    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-eef2d470-dockercfg-99nxt\""
Sep 04 23:48:29.609809 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.609791    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"]
Sep 04 23:48:29.650687 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.650662    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"]
Sep 04 23:48:29.653864 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.653850    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"
Sep 04 23:48:29.665373 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.665351    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"]
Sep 04 23:48:29.677698 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:48:29.677671    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7642-2d5d01\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="e4f71f3c-823d-4d8d-b200-dfa2f1c1c728" pod="opendatahub/test-curl-7642-2d5d01"
Sep 04 23:48:29.720012 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.719973    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1361a4f-55ba-44da-be3f-e5f56488bfb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb\" (UID: \"f1361a4f-55ba-44da-be3f-e5f56488bfb5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"
Sep 04 23:48:29.720103 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.720045    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkpdp\" (UniqueName: \"kubernetes.io/projected/f1361a4f-55ba-44da-be3f-e5f56488bfb5-kube-api-access-nkpdp\") pod \"payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb\" (UID: \"f1361a4f-55ba-44da-be3f-e5f56488bfb5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"
Sep 04 23:48:29.821287 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.821253    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2411ade-1405-4414-819c-e7bafe44e786-plugins-config-volume\") pod \"payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph\" (UID: \"b2411ade-1405-4414-819c-e7bafe44e786\") " pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"
Sep 04 23:48:29.821402 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.821357    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1361a4f-55ba-44da-be3f-e5f56488bfb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb\" (UID: \"f1361a4f-55ba-44da-be3f-e5f56488bfb5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"
Sep 04 23:48:29.821467 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.821411    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkpdp\" (UniqueName: \"kubernetes.io/projected/f1361a4f-55ba-44da-be3f-e5f56488bfb5-kube-api-access-nkpdp\") pod \"payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb\" (UID: \"f1361a4f-55ba-44da-be3f-e5f56488bfb5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"
Sep 04 23:48:29.821467 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.821445    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8mbx\" (UniqueName: \"kubernetes.io/projected/b2411ade-1405-4414-819c-e7bafe44e786-kube-api-access-z8mbx\") pod \"payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph\" (UID: \"b2411ade-1405-4414-819c-e7bafe44e786\") " pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"
Sep 04 23:48:29.822042 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.822021    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1361a4f-55ba-44da-be3f-e5f56488bfb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb\" (UID: \"f1361a4f-55ba-44da-be3f-e5f56488bfb5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"
Sep 04 23:48:29.830346 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.830327    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkpdp\" (UniqueName: \"kubernetes.io/projected/f1361a4f-55ba-44da-be3f-e5f56488bfb5-kube-api-access-nkpdp\") pod \"payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb\" (UID: \"f1361a4f-55ba-44da-be3f-e5f56488bfb5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"
Sep 04 23:48:29.912348 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.912319    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"
Sep 04 23:48:29.922519 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.922492    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8mbx\" (UniqueName: \"kubernetes.io/projected/b2411ade-1405-4414-819c-e7bafe44e786-kube-api-access-z8mbx\") pod \"payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph\" (UID: \"b2411ade-1405-4414-819c-e7bafe44e786\") " pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"
Sep 04 23:48:29.922608 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.922543    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2411ade-1405-4414-819c-e7bafe44e786-plugins-config-volume\") pod \"payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph\" (UID: \"b2411ade-1405-4414-819c-e7bafe44e786\") " pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"
Sep 04 23:48:29.923163 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.923144    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2411ade-1405-4414-819c-e7bafe44e786-plugins-config-volume\") pod \"payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph\" (UID: \"b2411ade-1405-4414-819c-e7bafe44e786\") " pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"
Sep 04 23:48:29.929826 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.929802    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8mbx\" (UniqueName: \"kubernetes.io/projected/b2411ade-1405-4414-819c-e7bafe44e786-kube-api-access-z8mbx\") pod \"payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph\" (UID: \"b2411ade-1405-4414-819c-e7bafe44e786\") " pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"
Sep 04 23:48:29.963975 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:29.963937    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"
Sep 04 23:48:30.174032 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.173968    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4f71f3c-823d-4d8d-b200-dfa2f1c1c728" path="/var/lib/kubelet/pods/e4f71f3c-823d-4d8d-b200-dfa2f1c1c728/volumes"
Sep 04 23:48:30.197315 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.197295    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"]
Sep 04 23:48:30.198861 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:48:30.198836    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1361a4f_55ba_44da_be3f_e5f56488bfb5.slice/crio-49ea2a7a62bdca4eec3737d93290d49258c6da2f795b7b0933ed020891159b89 WatchSource:0}: Error finding container 49ea2a7a62bdca4eec3737d93290d49258c6da2f795b7b0933ed020891159b89: Status 404 returned error can't find the container with id 49ea2a7a62bdca4eec3737d93290d49258c6da2f795b7b0933ed020891159b89
Sep 04 23:48:30.264005 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.263958    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"]
Sep 04 23:48:30.264978 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:48:30.264954    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2411ade_1405_4414_819c_e7bafe44e786.slice/crio-0dcf410e457fe981f5e92d8c2ae8b5470e330948b3a8c7ed0afa36521b0a74d0 WatchSource:0}: Error finding container 0dcf410e457fe981f5e92d8c2ae8b5470e330948b3a8c7ed0afa36521b0a74d0: Status 404 returned error can't find the container with id 0dcf410e457fe981f5e92d8c2ae8b5470e330948b3a8c7ed0afa36521b0a74d0
Sep 04 23:48:30.330704 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.330651    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph" event={"ID":"b2411ade-1405-4414-819c-e7bafe44e786","Type":"ContainerStarted","Data":"6d6568f656a56e09e865f466862c2d4dc46704fdcc56364581854ecd24227df4"}
Sep 04 23:48:30.330845 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.330719    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph" event={"ID":"b2411ade-1405-4414-819c-e7bafe44e786","Type":"ContainerStarted","Data":"0dcf410e457fe981f5e92d8c2ae8b5470e330948b3a8c7ed0afa36521b0a74d0"}
Sep 04 23:48:30.330845 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.330748    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"
Sep 04 23:48:30.333294 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.333274    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb" event={"ID":"f1361a4f-55ba-44da-be3f-e5f56488bfb5","Type":"ContainerStarted","Data":"8884e8e1768399de5e7c6900bd881009d87156e4c59cb23c47c41306ef8c8f39"}
Sep 04 23:48:30.333396 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.333312    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb" event={"ID":"f1361a4f-55ba-44da-be3f-e5f56488bfb5","Type":"ContainerStarted","Data":"49ea2a7a62bdca4eec3737d93290d49258c6da2f795b7b0933ed020891159b89"}
Sep 04 23:48:30.333396 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.333347    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"
Sep 04 23:48:30.348218 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.348166    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph" podStartSLOduration=1.348143348 podStartE2EDuration="1.348143348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:48:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:48:30.346225151 +0000 UTC m=+698.594225691" watchObservedRunningTime="2026-09-04 23:48:30.348143348 +0000 UTC m=+698.596143862"
Sep 04 23:48:30.361408 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.361353    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb" podStartSLOduration=1.361338021 podStartE2EDuration="1.361338021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:48:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:48:30.359956861 +0000 UTC m=+698.607957396" watchObservedRunningTime="2026-09-04 23:48:30.361338021 +0000 UTC m=+698.609338538"
Sep 04 23:48:30.502315 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.502236    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:30.564253 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.564119    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:30.724492 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:30.724458    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:31.066485 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.066420    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:31.160360 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.160330    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:31.204586 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.204554    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"]
Sep 04 23:48:31.208061 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.208038    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"
Sep 04 23:48:31.210734 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.210707    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-3c6688-dockercfg-jqr7s\""
Sep 04 23:48:31.210844 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.210707    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-3c6688\""
Sep 04 23:48:31.216311 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.216288    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"]
Sep 04 23:48:31.335574 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.335491    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gw4r\" (UniqueName: \"kubernetes.io/projected/d19c2acb-8c60-4edc-a8fb-8d953b007300-kube-api-access-8gw4r\") pod \"payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8\" (UID: \"d19c2acb-8c60-4edc-a8fb-8d953b007300\") " pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"
Sep 04 23:48:31.335574 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.335528    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d19c2acb-8c60-4edc-a8fb-8d953b007300-plugins-config-volume\") pod \"payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8\" (UID: \"d19c2acb-8c60-4edc-a8fb-8d953b007300\") " pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"
Sep 04 23:48:31.424443 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.424408    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:31.436527 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.436499    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gw4r\" (UniqueName: \"kubernetes.io/projected/d19c2acb-8c60-4edc-a8fb-8d953b007300-kube-api-access-8gw4r\") pod \"payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8\" (UID: \"d19c2acb-8c60-4edc-a8fb-8d953b007300\") " pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"
Sep 04 23:48:31.436671 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.436649    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d19c2acb-8c60-4edc-a8fb-8d953b007300-plugins-config-volume\") pod \"payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8\" (UID: \"d19c2acb-8c60-4edc-a8fb-8d953b007300\") " pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"
Sep 04 23:48:31.437412 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.437392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d19c2acb-8c60-4edc-a8fb-8d953b007300-plugins-config-volume\") pod \"payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8\" (UID: \"d19c2acb-8c60-4edc-a8fb-8d953b007300\") " pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"
Sep 04 23:48:31.444648 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.444629    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gw4r\" (UniqueName: \"kubernetes.io/projected/d19c2acb-8c60-4edc-a8fb-8d953b007300-kube-api-access-8gw4r\") pod \"payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8\" (UID: \"d19c2acb-8c60-4edc-a8fb-8d953b007300\") " pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"
Sep 04 23:48:31.520167 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.520132    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"
Sep 04 23:48:31.792098 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:31.792074    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"]
Sep 04 23:48:31.794168 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:48:31.794142    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd19c2acb_8c60_4edc_a8fb_8d953b007300.slice/crio-c2c5433545fa1717ce5b51e5b3d8614899bab19dabe9aa615df1c00c0b1e629b WatchSource:0}: Error finding container c2c5433545fa1717ce5b51e5b3d8614899bab19dabe9aa615df1c00c0b1e629b: Status 404 returned error can't find the container with id c2c5433545fa1717ce5b51e5b3d8614899bab19dabe9aa615df1c00c0b1e629b
Sep 04 23:48:32.189995 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:32.189947    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:32.344484 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:32.344448    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8" event={"ID":"d19c2acb-8c60-4edc-a8fb-8d953b007300","Type":"ContainerStarted","Data":"01ec6f8e43df226944dde8f9bbba302e041f6d1fa534b9ee00a6c267aef310bb"}
Sep 04 23:48:32.344664 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:32.344509    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8" event={"ID":"d19c2acb-8c60-4edc-a8fb-8d953b007300","Type":"ContainerStarted","Data":"c2c5433545fa1717ce5b51e5b3d8614899bab19dabe9aa615df1c00c0b1e629b"}
Sep 04 23:48:32.344664 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:32.344551    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"
Sep 04 23:48:32.360789 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:32.360740    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8" podStartSLOduration=1.360725989 podStartE2EDuration="1.360725989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:48:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:48:32.359424351 +0000 UTC m=+700.607424892" watchObservedRunningTime="2026-09-04 23:48:32.360725989 +0000 UTC m=+700.608726504"
Sep 04 23:48:34.296749 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:34.296719    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"
Sep 04 23:48:34.297212 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:34.296882    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"
Sep 04 23:48:35.422591 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:35.422550    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:35.579547 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:35.579509    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:35.668310 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:35.668236    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:35.768299 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:35.768227    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:37.209387 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:37.209351    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:38.424851 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:38.424799    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:39.692811 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:39.692782    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:39.793400 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:39.793372    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:39.915171 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:39.915139    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:40.018690 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:40.018600    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:41.340273 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:41.340240    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"
Sep 04 23:48:41.340747 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:41.340318    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"
Sep 04 23:48:42.197427 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:42.197398    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:42.934241 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:42.934166    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:43.351401 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:43.351316    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"
Sep 04 23:48:45.061836 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:45.061799    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:45.368506 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:45.368469    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:45.657205 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:45.657123    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:45.913799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:45.913723    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:46.014346 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:46.014289    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:46.087347 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:46.087317    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:46.266254 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:46.266174    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:48.437022 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:48.436983    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:48.536038 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:48.536004    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:49.600303 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:49.600260    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:49.701560 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:49.701526    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:50.097776 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.097735    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:50.185844 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.185809    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:50.189719 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.189695    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"]
Sep 04 23:48:50.196781 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.196762    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"
Sep 04 23:48:50.199352 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.199334    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-724e1f-dockercfg-nq64z\""
Sep 04 23:48:50.199465 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.199425    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-724e1f\""
Sep 04 23:48:50.203501 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.203478    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"]
Sep 04 23:48:50.218555 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.218531    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"]
Sep 04 23:48:50.229958 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.229887    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"
Sep 04 23:48:50.232777 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.232752    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"]
Sep 04 23:48:50.289959 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.289931    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a0bfc6d-d753-4c24-b760-3afb506c824e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n\" (UID: \"4a0bfc6d-d753-4c24-b760-3afb506c824e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"
Sep 04 23:48:50.290054 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.289975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7183cdd7-779c-434a-98e1-d9952bdf6ee9-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk\" (UID: \"7183cdd7-779c-434a-98e1-d9952bdf6ee9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"
Sep 04 23:48:50.290097 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.290048    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqdp8\" (UniqueName: \"kubernetes.io/projected/7183cdd7-779c-434a-98e1-d9952bdf6ee9-kube-api-access-tqdp8\") pod \"payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk\" (UID: \"7183cdd7-779c-434a-98e1-d9952bdf6ee9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"
Sep 04 23:48:50.290097 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.290082    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c25x2\" (UniqueName: \"kubernetes.io/projected/4a0bfc6d-d753-4c24-b760-3afb506c824e-kube-api-access-c25x2\") pod \"payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n\" (UID: \"4a0bfc6d-d753-4c24-b760-3afb506c824e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"
Sep 04 23:48:50.391499 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.391467    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a0bfc6d-d753-4c24-b760-3afb506c824e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n\" (UID: \"4a0bfc6d-d753-4c24-b760-3afb506c824e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"
Sep 04 23:48:50.391622 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.391541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7183cdd7-779c-434a-98e1-d9952bdf6ee9-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk\" (UID: \"7183cdd7-779c-434a-98e1-d9952bdf6ee9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"
Sep 04 23:48:50.391622 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.391611    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqdp8\" (UniqueName: \"kubernetes.io/projected/7183cdd7-779c-434a-98e1-d9952bdf6ee9-kube-api-access-tqdp8\") pod \"payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk\" (UID: \"7183cdd7-779c-434a-98e1-d9952bdf6ee9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"
Sep 04 23:48:50.391757 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.391644    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c25x2\" (UniqueName: \"kubernetes.io/projected/4a0bfc6d-d753-4c24-b760-3afb506c824e-kube-api-access-c25x2\") pod \"payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n\" (UID: \"4a0bfc6d-d753-4c24-b760-3afb506c824e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"
Sep 04 23:48:50.392259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.392239    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a0bfc6d-d753-4c24-b760-3afb506c824e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n\" (UID: \"4a0bfc6d-d753-4c24-b760-3afb506c824e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"
Sep 04 23:48:50.392355 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.392295    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7183cdd7-779c-434a-98e1-d9952bdf6ee9-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk\" (UID: \"7183cdd7-779c-434a-98e1-d9952bdf6ee9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"
Sep 04 23:48:50.400921 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.399885    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c25x2\" (UniqueName: \"kubernetes.io/projected/4a0bfc6d-d753-4c24-b760-3afb506c824e-kube-api-access-c25x2\") pod \"payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n\" (UID: \"4a0bfc6d-d753-4c24-b760-3afb506c824e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"
Sep 04 23:48:50.400921 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.400155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqdp8\" (UniqueName: \"kubernetes.io/projected/7183cdd7-779c-434a-98e1-d9952bdf6ee9-kube-api-access-tqdp8\") pod \"payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk\" (UID: \"7183cdd7-779c-434a-98e1-d9952bdf6ee9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"
Sep 04 23:48:50.508438 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.508409    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"
Sep 04 23:48:50.540842 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.540822    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"
Sep 04 23:48:50.619736 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.619693    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:50.779962 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.779941    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"]
Sep 04 23:48:50.781696 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:48:50.781668    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a0bfc6d_d753_4c24_b760_3afb506c824e.slice/crio-efcfae56723e3a41b54cfa1257138e3deabd43cb5d4217b4c8c10b2a3a78ee99 WatchSource:0}: Error finding container efcfae56723e3a41b54cfa1257138e3deabd43cb5d4217b4c8c10b2a3a78ee99: Status 404 returned error can't find the container with id efcfae56723e3a41b54cfa1257138e3deabd43cb5d4217b4c8c10b2a3a78ee99
Sep 04 23:48:50.823753 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:50.823731    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"]
Sep 04 23:48:50.825497 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:48:50.825473    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7183cdd7_779c_434a_98e1_d9952bdf6ee9.slice/crio-e1aca1106598f1f42581cf68f15ea637fb78bd1622adc5c8739a42cf4c9086f9 WatchSource:0}: Error finding container e1aca1106598f1f42581cf68f15ea637fb78bd1622adc5c8739a42cf4c9086f9: Status 404 returned error can't find the container with id e1aca1106598f1f42581cf68f15ea637fb78bd1622adc5c8739a42cf4c9086f9
Sep 04 23:48:51.416737 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:51.416702    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:51.442522 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:51.442490    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk" event={"ID":"7183cdd7-779c-434a-98e1-d9952bdf6ee9","Type":"ContainerStarted","Data":"c149315714a66abfc22bca92c98226f6ded4f8484e4efa0954210cd8c37bca47"}
Sep 04 23:48:51.442662 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:51.442542    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk" event={"ID":"7183cdd7-779c-434a-98e1-d9952bdf6ee9","Type":"ContainerStarted","Data":"e1aca1106598f1f42581cf68f15ea637fb78bd1622adc5c8739a42cf4c9086f9"}
Sep 04 23:48:51.442662 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:51.442603    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"
Sep 04 23:48:51.444184 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:51.444159    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n" event={"ID":"4a0bfc6d-d753-4c24-b760-3afb506c824e","Type":"ContainerStarted","Data":"cf131e49909d32f53a40e37d09cfbd26bb6701d7b662042cafe5b3bd6a9c844a"}
Sep 04 23:48:51.444312 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:51.444192    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n" event={"ID":"4a0bfc6d-d753-4c24-b760-3afb506c824e","Type":"ContainerStarted","Data":"efcfae56723e3a41b54cfa1257138e3deabd43cb5d4217b4c8c10b2a3a78ee99"}
Sep 04 23:48:51.444312 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:51.444212    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"
Sep 04 23:48:51.460708 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:51.460666    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk" podStartSLOduration=1.460653632 podStartE2EDuration="1.460653632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:48:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:48:51.459367219 +0000 UTC m=+719.707367759" watchObservedRunningTime="2026-09-04 23:48:51.460653632 +0000 UTC m=+719.708654142"
Sep 04 23:48:51.476102 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:51.476059    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n" podStartSLOduration=1.476046807 podStartE2EDuration="1.476046807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:48:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:48:51.4758608 +0000 UTC m=+719.723861343" watchObservedRunningTime="2026-09-04 23:48:51.476046807 +0000 UTC m=+719.724047305"
Sep 04 23:48:52.047176 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:52.047148    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:52.647622 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:52.647587    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:52.748968 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:52.748939    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:52.848608 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:52.848565    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:53.514512 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:53.514479    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:53.779455 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:53.779381    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:54.209066 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:54.208998    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:55.968710 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:55.968668    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:48:56.025339 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:48:56.025300    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:02.451084 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:02.451049    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"
Sep 04 23:49:02.451463 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:02.451133    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"
Sep 04 23:49:02.582206 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:02.582169    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:02.681087 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:02.681050    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:03.760699 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:03.760660    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:09.085701 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.085658    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:09.525611 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.525580    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:09.553293 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.553263    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"]
Sep 04 23:49:09.553518 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.553493    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph" podUID="b2411ade-1405-4414-819c-e7bafe44e786" containerName="payload-processing" containerID="cri-o://6d6568f656a56e09e865f466862c2d4dc46704fdcc56364581854ecd24227df4" gracePeriod=30
Sep 04 23:49:09.568342 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.568196    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"]
Sep 04 23:49:09.568464 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.568435    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb" podUID="f1361a4f-55ba-44da-be3f-e5f56488bfb5" containerName="payload-pre-processing" containerID="cri-o://8884e8e1768399de5e7c6900bd881009d87156e4c59cb23c47c41306ef8c8f39" gracePeriod=30
Sep 04 23:49:09.594229 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.594153    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:09.723527 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.723490    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:09.797437 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.797401    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:09.842082 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.842054    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"
Sep 04 23:49:09.844931 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.844893    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"
Sep 04 23:49:09.931793 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.931756    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:09.948532 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.948509    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1361a4f-55ba-44da-be3f-e5f56488bfb5-plugins-config-volume\") pod \"f1361a4f-55ba-44da-be3f-e5f56488bfb5\" (UID: \"f1361a4f-55ba-44da-be3f-e5f56488bfb5\") "
Sep 04 23:49:09.948629 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.948547    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2411ade-1405-4414-819c-e7bafe44e786-plugins-config-volume\") pod \"b2411ade-1405-4414-819c-e7bafe44e786\" (UID: \"b2411ade-1405-4414-819c-e7bafe44e786\") "
Sep 04 23:49:09.948629 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.948592    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8mbx\" (UniqueName: \"kubernetes.io/projected/b2411ade-1405-4414-819c-e7bafe44e786-kube-api-access-z8mbx\") pod \"b2411ade-1405-4414-819c-e7bafe44e786\" (UID: \"b2411ade-1405-4414-819c-e7bafe44e786\") "
Sep 04 23:49:09.948703 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.948639    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkpdp\" (UniqueName: \"kubernetes.io/projected/f1361a4f-55ba-44da-be3f-e5f56488bfb5-kube-api-access-nkpdp\") pod \"f1361a4f-55ba-44da-be3f-e5f56488bfb5\" (UID: \"f1361a4f-55ba-44da-be3f-e5f56488bfb5\") "
Sep 04 23:49:09.948970 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.948945    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1361a4f-55ba-44da-be3f-e5f56488bfb5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f1361a4f-55ba-44da-be3f-e5f56488bfb5" (UID: "f1361a4f-55ba-44da-be3f-e5f56488bfb5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:49:09.949021 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.948969    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2411ade-1405-4414-819c-e7bafe44e786-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2411ade-1405-4414-819c-e7bafe44e786" (UID: "b2411ade-1405-4414-819c-e7bafe44e786"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:49:09.950869 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.950849    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2411ade-1405-4414-819c-e7bafe44e786-kube-api-access-z8mbx" (OuterVolumeSpecName: "kube-api-access-z8mbx") pod "b2411ade-1405-4414-819c-e7bafe44e786" (UID: "b2411ade-1405-4414-819c-e7bafe44e786"). InnerVolumeSpecName "kube-api-access-z8mbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:49:09.950951 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:09.950929    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1361a4f-55ba-44da-be3f-e5f56488bfb5-kube-api-access-nkpdp" (OuterVolumeSpecName: "kube-api-access-nkpdp") pod "f1361a4f-55ba-44da-be3f-e5f56488bfb5" (UID: "f1361a4f-55ba-44da-be3f-e5f56488bfb5"). InnerVolumeSpecName "kube-api-access-nkpdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:49:10.049771 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.049712    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1361a4f-55ba-44da-be3f-e5f56488bfb5-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:49:10.049771 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.049732    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2411ade-1405-4414-819c-e7bafe44e786-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:49:10.049771 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.049740    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8mbx\" (UniqueName: \"kubernetes.io/projected/b2411ade-1405-4414-819c-e7bafe44e786-kube-api-access-z8mbx\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:49:10.049771 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.049750    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkpdp\" (UniqueName: \"kubernetes.io/projected/f1361a4f-55ba-44da-be3f-e5f56488bfb5-kube-api-access-nkpdp\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:49:10.356375 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.356297    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:10.457816 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.457778    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:10.545673 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.545639    2564 generic.go:356] "Generic (PLEG): container finished" podID="b2411ade-1405-4414-819c-e7bafe44e786" containerID="6d6568f656a56e09e865f466862c2d4dc46704fdcc56364581854ecd24227df4" exitCode=0
Sep 04 23:49:10.545803 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.545685    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"
Sep 04 23:49:10.545803 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.545725    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph" event={"ID":"b2411ade-1405-4414-819c-e7bafe44e786","Type":"ContainerDied","Data":"6d6568f656a56e09e865f466862c2d4dc46704fdcc56364581854ecd24227df4"}
Sep 04 23:49:10.545803 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.545781    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph" event={"ID":"b2411ade-1405-4414-819c-e7bafe44e786","Type":"ContainerDied","Data":"0dcf410e457fe981f5e92d8c2ae8b5470e330948b3a8c7ed0afa36521b0a74d0"}
Sep 04 23:49:10.545961 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.545808    2564 scope.go:117] "RemoveContainer" containerID="6d6568f656a56e09e865f466862c2d4dc46704fdcc56364581854ecd24227df4"
Sep 04 23:49:10.547571 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.547550    2564 generic.go:356] "Generic (PLEG): container finished" podID="f1361a4f-55ba-44da-be3f-e5f56488bfb5" containerID="8884e8e1768399de5e7c6900bd881009d87156e4c59cb23c47c41306ef8c8f39" exitCode=0
Sep 04 23:49:10.547694 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.547589    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb" event={"ID":"f1361a4f-55ba-44da-be3f-e5f56488bfb5","Type":"ContainerDied","Data":"8884e8e1768399de5e7c6900bd881009d87156e4c59cb23c47c41306ef8c8f39"}
Sep 04 23:49:10.547694 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.547617    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"
Sep 04 23:49:10.547694 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.547623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb" event={"ID":"f1361a4f-55ba-44da-be3f-e5f56488bfb5","Type":"ContainerDied","Data":"49ea2a7a62bdca4eec3737d93290d49258c6da2f795b7b0933ed020891159b89"}
Sep 04 23:49:10.555302 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.555153    2564 scope.go:117] "RemoveContainer" containerID="6d6568f656a56e09e865f466862c2d4dc46704fdcc56364581854ecd24227df4"
Sep 04 23:49:10.555466 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:49:10.555442    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d6568f656a56e09e865f466862c2d4dc46704fdcc56364581854ecd24227df4\": container with ID starting with 6d6568f656a56e09e865f466862c2d4dc46704fdcc56364581854ecd24227df4 not found: ID does not exist" containerID="6d6568f656a56e09e865f466862c2d4dc46704fdcc56364581854ecd24227df4"
Sep 04 23:49:10.555539 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.555479    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d6568f656a56e09e865f466862c2d4dc46704fdcc56364581854ecd24227df4"} err="failed to get container status \"6d6568f656a56e09e865f466862c2d4dc46704fdcc56364581854ecd24227df4\": rpc error: code = NotFound desc = could not find container \"6d6568f656a56e09e865f466862c2d4dc46704fdcc56364581854ecd24227df4\": container with ID starting with 6d6568f656a56e09e865f466862c2d4dc46704fdcc56364581854ecd24227df4 not found: ID does not exist"
Sep 04 23:49:10.555539 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.555501    2564 scope.go:117] "RemoveContainer" containerID="8884e8e1768399de5e7c6900bd881009d87156e4c59cb23c47c41306ef8c8f39"
Sep 04 23:49:10.556114 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.556038    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:10.563011 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.562992    2564 scope.go:117] "RemoveContainer" containerID="8884e8e1768399de5e7c6900bd881009d87156e4c59cb23c47c41306ef8c8f39"
Sep 04 23:49:10.563352 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:49:10.563331    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8884e8e1768399de5e7c6900bd881009d87156e4c59cb23c47c41306ef8c8f39\": container with ID starting with 8884e8e1768399de5e7c6900bd881009d87156e4c59cb23c47c41306ef8c8f39 not found: ID does not exist" containerID="8884e8e1768399de5e7c6900bd881009d87156e4c59cb23c47c41306ef8c8f39"
Sep 04 23:49:10.563439 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.563360    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8884e8e1768399de5e7c6900bd881009d87156e4c59cb23c47c41306ef8c8f39"} err="failed to get container status \"8884e8e1768399de5e7c6900bd881009d87156e4c59cb23c47c41306ef8c8f39\": rpc error: code = NotFound desc = could not find container \"8884e8e1768399de5e7c6900bd881009d87156e4c59cb23c47c41306ef8c8f39\": container with ID starting with 8884e8e1768399de5e7c6900bd881009d87156e4c59cb23c47c41306ef8c8f39 not found: ID does not exist"
Sep 04 23:49:10.565936 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.565919    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"]
Sep 04 23:49:10.570309 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.570289    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eef2d470-76954fc8db-gqgwb"]
Sep 04 23:49:10.581849 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.581829    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"]
Sep 04 23:49:10.588532 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.588512    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eef2d470-7b8b88b4fc-vgfph"]
Sep 04 23:49:10.613704 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.613684    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:10.906798 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:10.906714    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:11.065371 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:11.065336    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:12.180398 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:12.180364    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2411ade-1405-4414-819c-e7bafe44e786" path="/var/lib/kubelet/pods/b2411ade-1405-4414-819c-e7bafe44e786/volumes"
Sep 04 23:49:12.180760 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:12.180693    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1361a4f-55ba-44da-be3f-e5f56488bfb5" path="/var/lib/kubelet/pods/f1361a4f-55ba-44da-be3f-e5f56488bfb5/volumes"
Sep 04 23:49:13.111192 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.111161    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:13.195854 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.195824    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:13.276174 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.276126    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:13.547555 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.547448    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:13.723677 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.723640    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:13.795485 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.795451    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:13.800828 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.800763    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"]
Sep 04 23:49:13.801247 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.801230    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2411ade-1405-4414-819c-e7bafe44e786" containerName="payload-processing"
Sep 04 23:49:13.801338 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.801249    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2411ade-1405-4414-819c-e7bafe44e786" containerName="payload-processing"
Sep 04 23:49:13.801338 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.801267    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1361a4f-55ba-44da-be3f-e5f56488bfb5" containerName="payload-pre-processing"
Sep 04 23:49:13.801338 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.801275    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1361a4f-55ba-44da-be3f-e5f56488bfb5" containerName="payload-pre-processing"
Sep 04 23:49:13.801491 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.801370    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1361a4f-55ba-44da-be3f-e5f56488bfb5" containerName="payload-pre-processing"
Sep 04 23:49:13.801491 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.801386    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2411ade-1405-4414-819c-e7bafe44e786" containerName="payload-processing"
Sep 04 23:49:13.806179 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.806161    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"
Sep 04 23:49:13.809299 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.809283    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-8a8c53fa\""
Sep 04 23:49:13.821295 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.821274    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"]
Sep 04 23:49:13.853272 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.853247    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"]
Sep 04 23:49:13.856876 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.856856    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"
Sep 04 23:49:13.859600 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.859580    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-8a8c53fa-dockercfg-dtvbw\""
Sep 04 23:49:13.868063 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.868045    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"]
Sep 04 23:49:13.882519 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.882490    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzgst\" (UniqueName: \"kubernetes.io/projected/de91149c-4933-42c9-aac3-863889618dbb-kube-api-access-xzgst\") pod \"payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn\" (UID: \"de91149c-4933-42c9-aac3-863889618dbb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"
Sep 04 23:49:13.882632 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.882608    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de91149c-4933-42c9-aac3-863889618dbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn\" (UID: \"de91149c-4933-42c9-aac3-863889618dbb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"
Sep 04 23:49:13.983205 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.983166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de91149c-4933-42c9-aac3-863889618dbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn\" (UID: \"de91149c-4933-42c9-aac3-863889618dbb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"
Sep 04 23:49:13.983359 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.983237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e055d101-c302-4996-bc32-2268f6ffbab8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh\" (UID: \"e055d101-c302-4996-bc32-2268f6ffbab8\") " pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"
Sep 04 23:49:13.983359 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.983256    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7drj\" (UniqueName: \"kubernetes.io/projected/e055d101-c302-4996-bc32-2268f6ffbab8-kube-api-access-c7drj\") pod \"payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh\" (UID: \"e055d101-c302-4996-bc32-2268f6ffbab8\") " pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"
Sep 04 23:49:13.983359 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.983287    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzgst\" (UniqueName: \"kubernetes.io/projected/de91149c-4933-42c9-aac3-863889618dbb-kube-api-access-xzgst\") pod \"payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn\" (UID: \"de91149c-4933-42c9-aac3-863889618dbb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"
Sep 04 23:49:13.983864 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.983845    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de91149c-4933-42c9-aac3-863889618dbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn\" (UID: \"de91149c-4933-42c9-aac3-863889618dbb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"
Sep 04 23:49:13.990983 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:13.990956    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzgst\" (UniqueName: \"kubernetes.io/projected/de91149c-4933-42c9-aac3-863889618dbb-kube-api-access-xzgst\") pod \"payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn\" (UID: \"de91149c-4933-42c9-aac3-863889618dbb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"
Sep 04 23:49:14.084637 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.084575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e055d101-c302-4996-bc32-2268f6ffbab8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh\" (UID: \"e055d101-c302-4996-bc32-2268f6ffbab8\") " pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"
Sep 04 23:49:14.084637 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.084604    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7drj\" (UniqueName: \"kubernetes.io/projected/e055d101-c302-4996-bc32-2268f6ffbab8-kube-api-access-c7drj\") pod \"payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh\" (UID: \"e055d101-c302-4996-bc32-2268f6ffbab8\") " pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"
Sep 04 23:49:14.085212 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.085193    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e055d101-c302-4996-bc32-2268f6ffbab8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh\" (UID: \"e055d101-c302-4996-bc32-2268f6ffbab8\") " pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"
Sep 04 23:49:14.093113 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.093089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7drj\" (UniqueName: \"kubernetes.io/projected/e055d101-c302-4996-bc32-2268f6ffbab8-kube-api-access-c7drj\") pod \"payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh\" (UID: \"e055d101-c302-4996-bc32-2268f6ffbab8\") " pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"
Sep 04 23:49:14.115884 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.115858    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"
Sep 04 23:49:14.169910 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.169660    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"
Sep 04 23:49:14.386120 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.386097    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"]
Sep 04 23:49:14.387476 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:49:14.387454    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde91149c_4933_42c9_aac3_863889618dbb.slice/crio-651e4f7314aa36ad07f00fc1b2bb3310685040f706120fba3fe2a7ff6d44f2be WatchSource:0}: Error finding container 651e4f7314aa36ad07f00fc1b2bb3310685040f706120fba3fe2a7ff6d44f2be: Status 404 returned error can't find the container with id 651e4f7314aa36ad07f00fc1b2bb3310685040f706120fba3fe2a7ff6d44f2be
Sep 04 23:49:14.450825 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.450804    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"]
Sep 04 23:49:14.452779 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:49:14.452757    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode055d101_c302_4996_bc32_2268f6ffbab8.slice/crio-d00d51d45816b2fae68e70e0a6e7acdd0fa1f63ace16754e01a27bd73b08f09f WatchSource:0}: Error finding container d00d51d45816b2fae68e70e0a6e7acdd0fa1f63ace16754e01a27bd73b08f09f: Status 404 returned error can't find the container with id d00d51d45816b2fae68e70e0a6e7acdd0fa1f63ace16754e01a27bd73b08f09f
Sep 04 23:49:14.573486 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.573268    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn" event={"ID":"de91149c-4933-42c9-aac3-863889618dbb","Type":"ContainerStarted","Data":"8d1a5f8cf79962597395bd76e2596daf4ba39012c3449dda6279fdedbc7b4217"}
Sep 04 23:49:14.573486 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.573337    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn" event={"ID":"de91149c-4933-42c9-aac3-863889618dbb","Type":"ContainerStarted","Data":"651e4f7314aa36ad07f00fc1b2bb3310685040f706120fba3fe2a7ff6d44f2be"}
Sep 04 23:49:14.573677 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.573502    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"
Sep 04 23:49:14.575275 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.575249    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh" event={"ID":"e055d101-c302-4996-bc32-2268f6ffbab8","Type":"ContainerStarted","Data":"0bb6ba3d2d4e0071af30be13778fc5c1db285c5c5ee8c5105b972ee259ab020a"}
Sep 04 23:49:14.575382 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.575296    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh" event={"ID":"e055d101-c302-4996-bc32-2268f6ffbab8","Type":"ContainerStarted","Data":"d00d51d45816b2fae68e70e0a6e7acdd0fa1f63ace16754e01a27bd73b08f09f"}
Sep 04 23:49:14.575382 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.575317    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"
Sep 04 23:49:14.589325 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.589276    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn" podStartSLOduration=1.5892571709999999 podStartE2EDuration="1.589257171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:49:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:49:14.58846613 +0000 UTC m=+742.836466647" watchObservedRunningTime="2026-09-04 23:49:14.589257171 +0000 UTC m=+742.837257684"
Sep 04 23:49:14.610838 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:14.610792    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh" podStartSLOduration=1.6107746 podStartE2EDuration="1.6107746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:49:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:49:14.607978251 +0000 UTC m=+742.855978792" watchObservedRunningTime="2026-09-04 23:49:14.6107746 +0000 UTC m=+742.858775117"
Sep 04 23:49:25.582326 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:25.582294    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"
Sep 04 23:49:25.582782 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:25.582359    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"
Sep 04 23:49:31.450431 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:31.450391    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:44.698532 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:44.698480    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:44.713373 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:44.713345    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"]
Sep 04 23:49:44.713587 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:44.713562    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh" podUID="e055d101-c302-4996-bc32-2268f6ffbab8" containerName="payload-processing" containerID="cri-o://0bb6ba3d2d4e0071af30be13778fc5c1db285c5c5ee8c5105b972ee259ab020a" gracePeriod=30
Sep 04 23:49:44.722080 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:44.722026    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"]
Sep 04 23:49:44.722326 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:44.722263    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn" podUID="de91149c-4933-42c9-aac3-863889618dbb" containerName="payload-pre-processing" containerID="cri-o://8d1a5f8cf79962597395bd76e2596daf4ba39012c3449dda6279fdedbc7b4217" gracePeriod=30
Sep 04 23:49:44.798646 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:44.798619    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:44.868951 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:44.868890    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:44.927153 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:44.927117    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:45.007467 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.007432    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:45.026315 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.026049    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"
Sep 04 23:49:45.041697 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.041674    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7drj\" (UniqueName: \"kubernetes.io/projected/e055d101-c302-4996-bc32-2268f6ffbab8-kube-api-access-c7drj\") pod \"e055d101-c302-4996-bc32-2268f6ffbab8\" (UID: \"e055d101-c302-4996-bc32-2268f6ffbab8\") "
Sep 04 23:49:45.041806 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.041708    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e055d101-c302-4996-bc32-2268f6ffbab8-plugins-config-volume\") pod \"e055d101-c302-4996-bc32-2268f6ffbab8\" (UID: \"e055d101-c302-4996-bc32-2268f6ffbab8\") "
Sep 04 23:49:45.042108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.042088    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e055d101-c302-4996-bc32-2268f6ffbab8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e055d101-c302-4996-bc32-2268f6ffbab8" (UID: "e055d101-c302-4996-bc32-2268f6ffbab8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:49:45.043035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.043017    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"
Sep 04 23:49:45.044082 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.044026    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e055d101-c302-4996-bc32-2268f6ffbab8-kube-api-access-c7drj" (OuterVolumeSpecName: "kube-api-access-c7drj") pod "e055d101-c302-4996-bc32-2268f6ffbab8" (UID: "e055d101-c302-4996-bc32-2268f6ffbab8"). InnerVolumeSpecName "kube-api-access-c7drj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:49:45.142549 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.142520    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de91149c-4933-42c9-aac3-863889618dbb-plugins-config-volume\") pod \"de91149c-4933-42c9-aac3-863889618dbb\" (UID: \"de91149c-4933-42c9-aac3-863889618dbb\") "
Sep 04 23:49:45.142700 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.142599    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzgst\" (UniqueName: \"kubernetes.io/projected/de91149c-4933-42c9-aac3-863889618dbb-kube-api-access-xzgst\") pod \"de91149c-4933-42c9-aac3-863889618dbb\" (UID: \"de91149c-4933-42c9-aac3-863889618dbb\") "
Sep 04 23:49:45.142977 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.142955    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7drj\" (UniqueName: \"kubernetes.io/projected/e055d101-c302-4996-bc32-2268f6ffbab8-kube-api-access-c7drj\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:49:45.143083 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.142985    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e055d101-c302-4996-bc32-2268f6ffbab8-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:49:45.143083 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.142952    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de91149c-4933-42c9-aac3-863889618dbb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "de91149c-4933-42c9-aac3-863889618dbb" (UID: "de91149c-4933-42c9-aac3-863889618dbb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:49:45.144722 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.144693    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de91149c-4933-42c9-aac3-863889618dbb-kube-api-access-xzgst" (OuterVolumeSpecName: "kube-api-access-xzgst") pod "de91149c-4933-42c9-aac3-863889618dbb" (UID: "de91149c-4933-42c9-aac3-863889618dbb"). InnerVolumeSpecName "kube-api-access-xzgst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:49:45.243396 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.243361    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de91149c-4933-42c9-aac3-863889618dbb-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:49:45.243396 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.243392    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzgst\" (UniqueName: \"kubernetes.io/projected/de91149c-4933-42c9-aac3-863889618dbb-kube-api-access-xzgst\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:49:45.414203 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.414171    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:45.708477 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.708404    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:45.746073 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.746041    2564 generic.go:356] "Generic (PLEG): container finished" podID="de91149c-4933-42c9-aac3-863889618dbb" containerID="8d1a5f8cf79962597395bd76e2596daf4ba39012c3449dda6279fdedbc7b4217" exitCode=0
Sep 04 23:49:45.746210 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.746083    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"
Sep 04 23:49:45.746210 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.746127    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn" event={"ID":"de91149c-4933-42c9-aac3-863889618dbb","Type":"ContainerDied","Data":"8d1a5f8cf79962597395bd76e2596daf4ba39012c3449dda6279fdedbc7b4217"}
Sep 04 23:49:45.746210 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.746178    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn" event={"ID":"de91149c-4933-42c9-aac3-863889618dbb","Type":"ContainerDied","Data":"651e4f7314aa36ad07f00fc1b2bb3310685040f706120fba3fe2a7ff6d44f2be"}
Sep 04 23:49:45.746210 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.746205    2564 scope.go:117] "RemoveContainer" containerID="8d1a5f8cf79962597395bd76e2596daf4ba39012c3449dda6279fdedbc7b4217"
Sep 04 23:49:45.747608 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.747584    2564 generic.go:356] "Generic (PLEG): container finished" podID="e055d101-c302-4996-bc32-2268f6ffbab8" containerID="0bb6ba3d2d4e0071af30be13778fc5c1db285c5c5ee8c5105b972ee259ab020a" exitCode=0
Sep 04 23:49:45.747698 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.747620    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh" event={"ID":"e055d101-c302-4996-bc32-2268f6ffbab8","Type":"ContainerDied","Data":"0bb6ba3d2d4e0071af30be13778fc5c1db285c5c5ee8c5105b972ee259ab020a"}
Sep 04 23:49:45.747698 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.747626    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"
Sep 04 23:49:45.747698 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.747656    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh" event={"ID":"e055d101-c302-4996-bc32-2268f6ffbab8","Type":"ContainerDied","Data":"d00d51d45816b2fae68e70e0a6e7acdd0fa1f63ace16754e01a27bd73b08f09f"}
Sep 04 23:49:45.755404 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.755321    2564 scope.go:117] "RemoveContainer" containerID="8d1a5f8cf79962597395bd76e2596daf4ba39012c3449dda6279fdedbc7b4217"
Sep 04 23:49:45.755624 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:49:45.755597    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d1a5f8cf79962597395bd76e2596daf4ba39012c3449dda6279fdedbc7b4217\": container with ID starting with 8d1a5f8cf79962597395bd76e2596daf4ba39012c3449dda6279fdedbc7b4217 not found: ID does not exist" containerID="8d1a5f8cf79962597395bd76e2596daf4ba39012c3449dda6279fdedbc7b4217"
Sep 04 23:49:45.755716 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.755643    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d1a5f8cf79962597395bd76e2596daf4ba39012c3449dda6279fdedbc7b4217"} err="failed to get container status \"8d1a5f8cf79962597395bd76e2596daf4ba39012c3449dda6279fdedbc7b4217\": rpc error: code = NotFound desc = could not find container \"8d1a5f8cf79962597395bd76e2596daf4ba39012c3449dda6279fdedbc7b4217\": container with ID starting with 8d1a5f8cf79962597395bd76e2596daf4ba39012c3449dda6279fdedbc7b4217 not found: ID does not exist"
Sep 04 23:49:45.755716 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.755668    2564 scope.go:117] "RemoveContainer" containerID="0bb6ba3d2d4e0071af30be13778fc5c1db285c5c5ee8c5105b972ee259ab020a"
Sep 04 23:49:45.764531 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.764513    2564 scope.go:117] "RemoveContainer" containerID="0bb6ba3d2d4e0071af30be13778fc5c1db285c5c5ee8c5105b972ee259ab020a"
Sep 04 23:49:45.764823 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:49:45.764798    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bb6ba3d2d4e0071af30be13778fc5c1db285c5c5ee8c5105b972ee259ab020a\": container with ID starting with 0bb6ba3d2d4e0071af30be13778fc5c1db285c5c5ee8c5105b972ee259ab020a not found: ID does not exist" containerID="0bb6ba3d2d4e0071af30be13778fc5c1db285c5c5ee8c5105b972ee259ab020a"
Sep 04 23:49:45.764883 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.764832    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bb6ba3d2d4e0071af30be13778fc5c1db285c5c5ee8c5105b972ee259ab020a"} err="failed to get container status \"0bb6ba3d2d4e0071af30be13778fc5c1db285c5c5ee8c5105b972ee259ab020a\": rpc error: code = NotFound desc = could not find container \"0bb6ba3d2d4e0071af30be13778fc5c1db285c5c5ee8c5105b972ee259ab020a\": container with ID starting with 0bb6ba3d2d4e0071af30be13778fc5c1db285c5c5ee8c5105b972ee259ab020a not found: ID does not exist"
Sep 04 23:49:45.773844 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.773824    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"]
Sep 04 23:49:45.778169 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.778150    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-8a8c53fa-7c799d655d-gq9tn"]
Sep 04 23:49:45.793382 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.793363    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"]
Sep 04 23:49:45.800436 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:45.800416    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-8a8c53fa-56877f4674-mw5jh"]
Sep 04 23:49:46.175121 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:46.175090    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de91149c-4933-42c9-aac3-863889618dbb" path="/var/lib/kubelet/pods/de91149c-4933-42c9-aac3-863889618dbb/volumes"
Sep 04 23:49:46.175450 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:46.175437    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e055d101-c302-4996-bc32-2268f6ffbab8" path="/var/lib/kubelet/pods/e055d101-c302-4996-bc32-2268f6ffbab8/volumes"
Sep 04 23:49:46.523007 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:46.522928    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:47.792796 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:47.792760    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:47.938874 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:47.938843    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:48.942108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:48.942077    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:49.042887 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.042853    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:49.144543 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.144508    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:49.412178 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.412135    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:49.509826 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.509799    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:49:49.511263 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.511242    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"]
Sep 04 23:49:49.511616 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.511604    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e055d101-c302-4996-bc32-2268f6ffbab8" containerName="payload-processing"
Sep 04 23:49:49.511653 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.511618    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e055d101-c302-4996-bc32-2268f6ffbab8" containerName="payload-processing"
Sep 04 23:49:49.511653 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.511640    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de91149c-4933-42c9-aac3-863889618dbb" containerName="payload-pre-processing"
Sep 04 23:49:49.511653 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.511645    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="de91149c-4933-42c9-aac3-863889618dbb" containerName="payload-pre-processing"
Sep 04 23:49:49.511781 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.511703    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e055d101-c302-4996-bc32-2268f6ffbab8" containerName="payload-processing"
Sep 04 23:49:49.511781 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.511711    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="de91149c-4933-42c9-aac3-863889618dbb" containerName="payload-pre-processing"
Sep 04 23:49:49.514583 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.514568    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"
Sep 04 23:49:49.518184 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.518156    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b52c7b74-dockercfg-xxpg8\""
Sep 04 23:49:49.518284 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.518184    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b52c7b74\""
Sep 04 23:49:49.529085 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.529061    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"]
Sep 04 23:49:49.584551 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.584527    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vrbb\" (UniqueName: \"kubernetes.io/projected/fc87c65c-0064-494f-804b-a6814bdff060-kube-api-access-5vrbb\") pod \"payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42\" (UID: \"fc87c65c-0064-494f-804b-a6814bdff060\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"
Sep 04 23:49:49.584674 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.584561    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc87c65c-0064-494f-804b-a6814bdff060-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42\" (UID: \"fc87c65c-0064-494f-804b-a6814bdff060\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"
Sep 04 23:49:49.685094 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.684999    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vrbb\" (UniqueName: \"kubernetes.io/projected/fc87c65c-0064-494f-804b-a6814bdff060-kube-api-access-5vrbb\") pod \"payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42\" (UID: \"fc87c65c-0064-494f-804b-a6814bdff060\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"
Sep 04 23:49:49.685094 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.685062    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc87c65c-0064-494f-804b-a6814bdff060-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42\" (UID: \"fc87c65c-0064-494f-804b-a6814bdff060\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"
Sep 04 23:49:49.685932 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.685891    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc87c65c-0064-494f-804b-a6814bdff060-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42\" (UID: \"fc87c65c-0064-494f-804b-a6814bdff060\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"
Sep 04 23:49:49.692672 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.692646    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vrbb\" (UniqueName: \"kubernetes.io/projected/fc87c65c-0064-494f-804b-a6814bdff060-kube-api-access-5vrbb\") pod \"payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42\" (UID: \"fc87c65c-0064-494f-804b-a6814bdff060\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"
Sep 04 23:49:49.824802 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:49.824768    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"
Sep 04 23:49:50.102225 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:50.102198    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"]
Sep 04 23:49:50.104227 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:49:50.104195    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc87c65c_0064_494f_804b_a6814bdff060.slice/crio-94e18cb18561bdaee18e22f5d86fc5315ad59c99522cc768bfe12f26de389625 WatchSource:0}: Error finding container 94e18cb18561bdaee18e22f5d86fc5315ad59c99522cc768bfe12f26de389625: Status 404 returned error can't find the container with id 94e18cb18561bdaee18e22f5d86fc5315ad59c99522cc768bfe12f26de389625
Sep 04 23:49:50.778303 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:50.778265    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42" event={"ID":"fc87c65c-0064-494f-804b-a6814bdff060","Type":"ContainerStarted","Data":"8cf3cba1c08422a9ac7980d4cfbee1cdc0a8c70dc7a88f042dafad96c8c9802a"}
Sep 04 23:49:50.778462 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:50.778319    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42" event={"ID":"fc87c65c-0064-494f-804b-a6814bdff060","Type":"ContainerStarted","Data":"94e18cb18561bdaee18e22f5d86fc5315ad59c99522cc768bfe12f26de389625"}
Sep 04 23:49:50.778462 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:50.778344    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"
Sep 04 23:49:50.793581 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:49:50.793540    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42" podStartSLOduration=1.793526768 podStartE2EDuration="1.793526768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:49:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:49:50.792285618 +0000 UTC m=+779.040286154" watchObservedRunningTime="2026-09-04 23:49:50.793526768 +0000 UTC m=+779.041527284"
Sep 04 23:50:01.785944 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:01.785879    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"
Sep 04 23:50:12.516649 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.516617    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"]
Sep 04 23:50:12.522065 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.522044    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.526281 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.526257    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-3c6688\"/\"default-dockercfg-c6sr4\""
Sep 04 23:50:12.526636 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.526615    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-3c6688\"/\"autores-model-3c6688-kserve-self-signed-certs\""
Sep 04 23:50:12.526916 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.526876    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-3c6688\"/\"kube-root-ca.crt\""
Sep 04 23:50:12.527055 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.527037    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-3c6688\"/\"openshift-service-ca.crt\""
Sep 04 23:50:12.527509 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.527487    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"]
Sep 04 23:50:12.550410 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.550379    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:12.696032 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.696006    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-tmp-dir\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.696190 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.696041    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-home\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.696190 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.696068    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-model-cache\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.696190 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.696085    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-tls-certs\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.696190 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.696155    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7kgq\" (UniqueName: \"kubernetes.io/projected/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-kube-api-access-x7kgq\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.696326 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.696241    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-dshm\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.796953 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.796850    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-tmp-dir\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.796953 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.796884    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-home\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.796953 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.796940    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-model-cache\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.797214 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.796958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-tls-certs\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.797214 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.796981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7kgq\" (UniqueName: \"kubernetes.io/projected/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-kube-api-access-x7kgq\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.797214 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.797079    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-dshm\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.797380 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.797318    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-tmp-dir\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.797380 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.797340    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-model-cache\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.797460 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.797378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-home\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.799261 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.799234    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-dshm\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.799636 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.799618    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-tls-certs\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.804302 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.804285    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7kgq\" (UniqueName: \"kubernetes.io/projected/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-kube-api-access-x7kgq\") pod \"autores-model-3c6688-kserve-557cf8fb75-wnm69\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") " pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:12.833342 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:12.833320    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:13.107613 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:13.107591    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"]
Sep 04 23:50:13.109889 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:50:13.109862    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad5db71a_d046_40b1_bbc5_48b6333ef4dd.slice/crio-14705afed70db7d2a58cf567bce141b7cc151af293b3c5f3a63f3d04f0a2abcc WatchSource:0}: Error finding container 14705afed70db7d2a58cf567bce141b7cc151af293b3c5f3a63f3d04f0a2abcc: Status 404 returned error can't find the container with id 14705afed70db7d2a58cf567bce141b7cc151af293b3c5f3a63f3d04f0a2abcc
Sep 04 23:50:13.899989 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:13.899882    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69" event={"ID":"ad5db71a-d046-40b1-bbc5-48b6333ef4dd","Type":"ContainerStarted","Data":"10ae65f48fad2eecdc6dc6807ca2b952b88465019af8fe4922ed6b8cbb71a5a8"}
Sep 04 23:50:13.899989 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:13.899958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69" event={"ID":"ad5db71a-d046-40b1-bbc5-48b6333ef4dd","Type":"ContainerStarted","Data":"14705afed70db7d2a58cf567bce141b7cc151af293b3c5f3a63f3d04f0a2abcc"}
Sep 04 23:50:13.918951 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:13.917728    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69" podStartSLOduration=1.917709051 podStartE2EDuration="1.917709051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:50:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:50:13.915135432 +0000 UTC m=+802.163135963" watchObservedRunningTime="2026-09-04 23:50:13.917709051 +0000 UTC m=+802.165709570"
Sep 04 23:50:19.417951 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.417916    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"]
Sep 04 23:50:19.418404 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.418136    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42" podUID="fc87c65c-0064-494f-804b-a6814bdff060" containerName="payload-pre-processing" containerID="cri-o://8cf3cba1c08422a9ac7980d4cfbee1cdc0a8c70dc7a88f042dafad96c8c9802a" gracePeriod=30
Sep 04 23:50:19.440780 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.440742    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:19.539163 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.539132    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:19.594594 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.594550    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:19.673957 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.673870    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"
Sep 04 23:50:19.861726 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.861691    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vrbb\" (UniqueName: \"kubernetes.io/projected/fc87c65c-0064-494f-804b-a6814bdff060-kube-api-access-5vrbb\") pod \"fc87c65c-0064-494f-804b-a6814bdff060\" (UID: \"fc87c65c-0064-494f-804b-a6814bdff060\") "
Sep 04 23:50:19.861945 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.861780    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc87c65c-0064-494f-804b-a6814bdff060-plugins-config-volume\") pod \"fc87c65c-0064-494f-804b-a6814bdff060\" (UID: \"fc87c65c-0064-494f-804b-a6814bdff060\") "
Sep 04 23:50:19.862225 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.862194    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc87c65c-0064-494f-804b-a6814bdff060-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fc87c65c-0064-494f-804b-a6814bdff060" (UID: "fc87c65c-0064-494f-804b-a6814bdff060"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:50:19.864111 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.864082    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc87c65c-0064-494f-804b-a6814bdff060-kube-api-access-5vrbb" (OuterVolumeSpecName: "kube-api-access-5vrbb") pod "fc87c65c-0064-494f-804b-a6814bdff060" (UID: "fc87c65c-0064-494f-804b-a6814bdff060"). InnerVolumeSpecName "kube-api-access-5vrbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:50:19.930568 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.930500    2564 generic.go:356] "Generic (PLEG): container finished" podID="fc87c65c-0064-494f-804b-a6814bdff060" containerID="8cf3cba1c08422a9ac7980d4cfbee1cdc0a8c70dc7a88f042dafad96c8c9802a" exitCode=0
Sep 04 23:50:19.930568 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.930549    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"
Sep 04 23:50:19.930744 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.930584    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42" event={"ID":"fc87c65c-0064-494f-804b-a6814bdff060","Type":"ContainerDied","Data":"8cf3cba1c08422a9ac7980d4cfbee1cdc0a8c70dc7a88f042dafad96c8c9802a"}
Sep 04 23:50:19.930744 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.930637    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42" event={"ID":"fc87c65c-0064-494f-804b-a6814bdff060","Type":"ContainerDied","Data":"94e18cb18561bdaee18e22f5d86fc5315ad59c99522cc768bfe12f26de389625"}
Sep 04 23:50:19.930744 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.930659    2564 scope.go:117] "RemoveContainer" containerID="8cf3cba1c08422a9ac7980d4cfbee1cdc0a8c70dc7a88f042dafad96c8c9802a"
Sep 04 23:50:19.946560 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.946539    2564 scope.go:117] "RemoveContainer" containerID="8cf3cba1c08422a9ac7980d4cfbee1cdc0a8c70dc7a88f042dafad96c8c9802a"
Sep 04 23:50:19.946850 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:50:19.946830    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cf3cba1c08422a9ac7980d4cfbee1cdc0a8c70dc7a88f042dafad96c8c9802a\": container with ID starting with 8cf3cba1c08422a9ac7980d4cfbee1cdc0a8c70dc7a88f042dafad96c8c9802a not found: ID does not exist" containerID="8cf3cba1c08422a9ac7980d4cfbee1cdc0a8c70dc7a88f042dafad96c8c9802a"
Sep 04 23:50:19.946954 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.946856    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cf3cba1c08422a9ac7980d4cfbee1cdc0a8c70dc7a88f042dafad96c8c9802a"} err="failed to get container status \"8cf3cba1c08422a9ac7980d4cfbee1cdc0a8c70dc7a88f042dafad96c8c9802a\": rpc error: code = NotFound desc = could not find container \"8cf3cba1c08422a9ac7980d4cfbee1cdc0a8c70dc7a88f042dafad96c8c9802a\": container with ID starting with 8cf3cba1c08422a9ac7980d4cfbee1cdc0a8c70dc7a88f042dafad96c8c9802a not found: ID does not exist"
Sep 04 23:50:19.957241 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.957219    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"]
Sep 04 23:50:19.960751 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.960730    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b52c7b74-78577d474b-ksx42"]
Sep 04 23:50:19.963369 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.963348    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vrbb\" (UniqueName: \"kubernetes.io/projected/fc87c65c-0064-494f-804b-a6814bdff060-kube-api-access-5vrbb\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:50:19.963369 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:19.963369    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc87c65c-0064-494f-804b-a6814bdff060-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:50:20.174951 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:20.174920    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc87c65c-0064-494f-804b-a6814bdff060" path="/var/lib/kubelet/pods/fc87c65c-0064-494f-804b-a6814bdff060/volumes"
Sep 04 23:50:20.628914 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:20.628854    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:21.068997 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:21.068918    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:21.307470 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:21.307429    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:21.415028 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:21.414991    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:22.834515 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:22.834482    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:22.834932 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:22.834531    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:22.847166 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:22.847140    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:22.959366 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:22.959332    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:23.414744 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:23.414715    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:23.513850 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:23.513784    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:24.586028 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:24.585985    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:24.689876 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:24.689828    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:24.885679 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:24.885631    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:25.080553 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.080520    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:25.244209 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.244120    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"]
Sep 04 23:50:25.244600 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.244582    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc87c65c-0064-494f-804b-a6814bdff060" containerName="payload-pre-processing"
Sep 04 23:50:25.244687 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.244605    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc87c65c-0064-494f-804b-a6814bdff060" containerName="payload-pre-processing"
Sep 04 23:50:25.244741 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.244698    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc87c65c-0064-494f-804b-a6814bdff060" containerName="payload-pre-processing"
Sep 04 23:50:25.249237 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.249216    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"
Sep 04 23:50:25.253581 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.253562    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-58fce2d8-dockercfg-q764j\""
Sep 04 23:50:25.253930 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.253909    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-58fce2d8\""
Sep 04 23:50:25.260788 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.260771    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"]
Sep 04 23:50:25.278804 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.278775    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:25.312190 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.312158    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a3cae7-ea93-49f8-b5aa-6c78a779cc96-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv\" (UID: \"58a3cae7-ea93-49f8-b5aa-6c78a779cc96\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"
Sep 04 23:50:25.312320 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.312204    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdhrd\" (UniqueName: \"kubernetes.io/projected/58a3cae7-ea93-49f8-b5aa-6c78a779cc96-kube-api-access-kdhrd\") pod \"payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv\" (UID: \"58a3cae7-ea93-49f8-b5aa-6c78a779cc96\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"
Sep 04 23:50:25.413178 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.413139    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a3cae7-ea93-49f8-b5aa-6c78a779cc96-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv\" (UID: \"58a3cae7-ea93-49f8-b5aa-6c78a779cc96\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"
Sep 04 23:50:25.413320 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.413196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdhrd\" (UniqueName: \"kubernetes.io/projected/58a3cae7-ea93-49f8-b5aa-6c78a779cc96-kube-api-access-kdhrd\") pod \"payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv\" (UID: \"58a3cae7-ea93-49f8-b5aa-6c78a779cc96\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"
Sep 04 23:50:25.413848 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.413829    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a3cae7-ea93-49f8-b5aa-6c78a779cc96-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv\" (UID: \"58a3cae7-ea93-49f8-b5aa-6c78a779cc96\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"
Sep 04 23:50:25.420952 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.420920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdhrd\" (UniqueName: \"kubernetes.io/projected/58a3cae7-ea93-49f8-b5aa-6c78a779cc96-kube-api-access-kdhrd\") pod \"payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv\" (UID: \"58a3cae7-ea93-49f8-b5aa-6c78a779cc96\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"
Sep 04 23:50:25.559207 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.559110    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"
Sep 04 23:50:25.833173 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.833149    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"]
Sep 04 23:50:25.834957 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:50:25.834930    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a3cae7_ea93_49f8_b5aa_6c78a779cc96.slice/crio-5081762258b5f1b4bc220106cf13be73431037e8d19734e84a1e1eeecf58da91 WatchSource:0}: Error finding container 5081762258b5f1b4bc220106cf13be73431037e8d19734e84a1e1eeecf58da91: Status 404 returned error can't find the container with id 5081762258b5f1b4bc220106cf13be73431037e8d19734e84a1e1eeecf58da91
Sep 04 23:50:25.966063 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.966014    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv" event={"ID":"58a3cae7-ea93-49f8-b5aa-6c78a779cc96","Type":"ContainerStarted","Data":"dc48442f9de64a3bbd903046cb706be0ae4981e0752b4eea3a39c27799056e99"}
Sep 04 23:50:25.966264 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.966084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv" event={"ID":"58a3cae7-ea93-49f8-b5aa-6c78a779cc96","Type":"ContainerStarted","Data":"5081762258b5f1b4bc220106cf13be73431037e8d19734e84a1e1eeecf58da91"}
Sep 04 23:50:25.966458 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.966427    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"
Sep 04 23:50:25.982090 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:25.982039    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv" podStartSLOduration=0.98202108 podStartE2EDuration="982.02108ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:50:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:50:25.98092958 +0000 UTC m=+814.228930124" watchObservedRunningTime="2026-09-04 23:50:25.98202108 +0000 UTC m=+814.230021596"
Sep 04 23:50:36.974237 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:36.974204    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"
Sep 04 23:50:39.101051 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.101018    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"]
Sep 04 23:50:39.104778 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.104761    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.108483 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.108459    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d8b443\"/\"default-dockercfg-ln9xh\""
Sep 04 23:50:39.108614 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.108517    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d8b443\"/\"openshift-service-ca.crt\""
Sep 04 23:50:39.108614 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.108519    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d8b443\"/\"test-model-d8b443-kserve-self-signed-certs\""
Sep 04 23:50:39.108614 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.108471    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d8b443\"/\"kube-root-ca.crt\""
Sep 04 23:50:39.112271 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.112250    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"]
Sep 04 23:50:39.239958 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.239920    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgqmk\" (UniqueName: \"kubernetes.io/projected/faf89f99-d54b-4757-a109-4d752bea81db-kube-api-access-wgqmk\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.240114 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.239967    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-home\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.240114 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.240011    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faf89f99-d54b-4757-a109-4d752bea81db-tls-certs\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.240114 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.240056    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-tmp-dir\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.240226 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.240135    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-model-cache\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.240226 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.240183    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-dshm\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.341526 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.341492    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-dshm\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.341695 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.341539    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgqmk\" (UniqueName: \"kubernetes.io/projected/faf89f99-d54b-4757-a109-4d752bea81db-kube-api-access-wgqmk\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.341695 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.341561    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-home\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.341695 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.341591    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faf89f99-d54b-4757-a109-4d752bea81db-tls-certs\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.341695 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.341615    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-tmp-dir\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.341695 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.341645    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-model-cache\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.342060 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.342036    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-model-cache\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.342130 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.342058    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-home\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.342130 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.342103    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-tmp-dir\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.343872 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.343849    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-dshm\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.344228 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.344212    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faf89f99-d54b-4757-a109-4d752bea81db-tls-certs\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.349214 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.349191    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgqmk\" (UniqueName: \"kubernetes.io/projected/faf89f99-d54b-4757-a109-4d752bea81db-kube-api-access-wgqmk\") pod \"test-model-d8b443-kserve-748bdd646c-gvp5p\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") " pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.414511 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.414485    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:39.415445 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.415416    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:39.684057 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:39.684033    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"]
Sep 04 23:50:39.685565 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:50:39.685540    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfaf89f99_d54b_4757_a109_4d752bea81db.slice/crio-fddba322fe11df8f57f795032f5196cac7db0756eba8c596c281dc64003ef39f WatchSource:0}: Error finding container fddba322fe11df8f57f795032f5196cac7db0756eba8c596c281dc64003ef39f: Status 404 returned error can't find the container with id fddba322fe11df8f57f795032f5196cac7db0756eba8c596c281dc64003ef39f
Sep 04 23:50:40.039284 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:40.039200    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p" event={"ID":"faf89f99-d54b-4757-a109-4d752bea81db","Type":"ContainerStarted","Data":"3a900991933dc7669ed12e4afe2d684ff3fc62231955ce6c578b77556503e00c"}
Sep 04 23:50:40.039284 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:40.039239    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p" event={"ID":"faf89f99-d54b-4757-a109-4d752bea81db","Type":"ContainerStarted","Data":"fddba322fe11df8f57f795032f5196cac7db0756eba8c596c281dc64003ef39f"}
Sep 04 23:50:40.055980 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:40.055937    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p" podStartSLOduration=1.055922029 podStartE2EDuration="1.055922029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:50:40.054521952 +0000 UTC m=+828.302522488" watchObservedRunningTime="2026-09-04 23:50:40.055922029 +0000 UTC m=+828.303922541"
Sep 04 23:50:42.120493 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:42.120454    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:43.054045 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:43.053950    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:44.036230 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:44.036190    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"]
Sep 04 23:50:44.056092 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:44.055921    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:44.286938 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:44.286831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:45.596375 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:45.596339    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:49.415368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:49.415339    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:49.415745 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:49.415376    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:49.427966 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:49.427945    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:50.108219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:50.108191    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:50:54.286070 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:54.286033    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:54.931000 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:54.930957    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"]
Sep 04 23:50:54.931198 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:54.931176    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv" podUID="58a3cae7-ea93-49f8-b5aa-6c78a779cc96" containerName="payload-pre-processing" containerID="cri-o://dc48442f9de64a3bbd903046cb706be0ae4981e0752b4eea3a39c27799056e99" gracePeriod=30
Sep 04 23:50:55.066448 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.062878    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"]
Sep 04 23:50:55.066448 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.063216    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8" podUID="d19c2acb-8c60-4edc-a8fb-8d953b007300" containerName="payload-processing" containerID="cri-o://01ec6f8e43df226944dde8f9bbba302e041f6d1fa534b9ee00a6c267aef310bb" gracePeriod=30
Sep 04 23:50:55.122595 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.122566    2564 generic.go:356] "Generic (PLEG): container finished" podID="58a3cae7-ea93-49f8-b5aa-6c78a779cc96" containerID="dc48442f9de64a3bbd903046cb706be0ae4981e0752b4eea3a39c27799056e99" exitCode=0
Sep 04 23:50:55.122730 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.122636    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv" event={"ID":"58a3cae7-ea93-49f8-b5aa-6c78a779cc96","Type":"ContainerDied","Data":"dc48442f9de64a3bbd903046cb706be0ae4981e0752b4eea3a39c27799056e99"}
Sep 04 23:50:55.196920 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.196873    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"
Sep 04 23:50:55.239193 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.239138    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:55.292953 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.292927    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdhrd\" (UniqueName: \"kubernetes.io/projected/58a3cae7-ea93-49f8-b5aa-6c78a779cc96-kube-api-access-kdhrd\") pod \"58a3cae7-ea93-49f8-b5aa-6c78a779cc96\" (UID: \"58a3cae7-ea93-49f8-b5aa-6c78a779cc96\") "
Sep 04 23:50:55.293385 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.293030    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a3cae7-ea93-49f8-b5aa-6c78a779cc96-plugins-config-volume\") pod \"58a3cae7-ea93-49f8-b5aa-6c78a779cc96\" (UID: \"58a3cae7-ea93-49f8-b5aa-6c78a779cc96\") "
Sep 04 23:50:55.293533 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.293508    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58a3cae7-ea93-49f8-b5aa-6c78a779cc96-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "58a3cae7-ea93-49f8-b5aa-6c78a779cc96" (UID: "58a3cae7-ea93-49f8-b5aa-6c78a779cc96"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:50:55.294209 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.294183    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:55.295335 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.295313    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58a3cae7-ea93-49f8-b5aa-6c78a779cc96-kube-api-access-kdhrd" (OuterVolumeSpecName: "kube-api-access-kdhrd") pod "58a3cae7-ea93-49f8-b5aa-6c78a779cc96" (UID: "58a3cae7-ea93-49f8-b5aa-6c78a779cc96"). InnerVolumeSpecName "kube-api-access-kdhrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:50:55.337517 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.337496    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"
Sep 04 23:50:55.394353 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.394315    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a3cae7-ea93-49f8-b5aa-6c78a779cc96-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:50:55.394353 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.394344    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdhrd\" (UniqueName: \"kubernetes.io/projected/58a3cae7-ea93-49f8-b5aa-6c78a779cc96-kube-api-access-kdhrd\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:50:55.495650 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.495585    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gw4r\" (UniqueName: \"kubernetes.io/projected/d19c2acb-8c60-4edc-a8fb-8d953b007300-kube-api-access-8gw4r\") pod \"d19c2acb-8c60-4edc-a8fb-8d953b007300\" (UID: \"d19c2acb-8c60-4edc-a8fb-8d953b007300\") "
Sep 04 23:50:55.495650 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.495624    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d19c2acb-8c60-4edc-a8fb-8d953b007300-plugins-config-volume\") pod \"d19c2acb-8c60-4edc-a8fb-8d953b007300\" (UID: \"d19c2acb-8c60-4edc-a8fb-8d953b007300\") "
Sep 04 23:50:55.496068 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.496035    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d19c2acb-8c60-4edc-a8fb-8d953b007300-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d19c2acb-8c60-4edc-a8fb-8d953b007300" (UID: "d19c2acb-8c60-4edc-a8fb-8d953b007300"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:50:55.497716 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.497695    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d19c2acb-8c60-4edc-a8fb-8d953b007300-kube-api-access-8gw4r" (OuterVolumeSpecName: "kube-api-access-8gw4r") pod "d19c2acb-8c60-4edc-a8fb-8d953b007300" (UID: "d19c2acb-8c60-4edc-a8fb-8d953b007300"). InnerVolumeSpecName "kube-api-access-8gw4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:50:55.596950 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.596922    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gw4r\" (UniqueName: \"kubernetes.io/projected/d19c2acb-8c60-4edc-a8fb-8d953b007300-kube-api-access-8gw4r\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:50:55.596950 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.596947    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d19c2acb-8c60-4edc-a8fb-8d953b007300-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:50:55.658967 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:55.658938    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:56.128708 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.128672    2564 generic.go:356] "Generic (PLEG): container finished" podID="d19c2acb-8c60-4edc-a8fb-8d953b007300" containerID="01ec6f8e43df226944dde8f9bbba302e041f6d1fa534b9ee00a6c267aef310bb" exitCode=0
Sep 04 23:50:56.128884 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.128724    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"
Sep 04 23:50:56.128884 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.128716    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8" event={"ID":"d19c2acb-8c60-4edc-a8fb-8d953b007300","Type":"ContainerDied","Data":"01ec6f8e43df226944dde8f9bbba302e041f6d1fa534b9ee00a6c267aef310bb"}
Sep 04 23:50:56.128884 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.128847    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8" event={"ID":"d19c2acb-8c60-4edc-a8fb-8d953b007300","Type":"ContainerDied","Data":"c2c5433545fa1717ce5b51e5b3d8614899bab19dabe9aa615df1c00c0b1e629b"}
Sep 04 23:50:56.128884 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.128872    2564 scope.go:117] "RemoveContainer" containerID="01ec6f8e43df226944dde8f9bbba302e041f6d1fa534b9ee00a6c267aef310bb"
Sep 04 23:50:56.130303 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.130285    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"
Sep 04 23:50:56.130303 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.130295    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv" event={"ID":"58a3cae7-ea93-49f8-b5aa-6c78a779cc96","Type":"ContainerDied","Data":"5081762258b5f1b4bc220106cf13be73431037e8d19734e84a1e1eeecf58da91"}
Sep 04 23:50:56.139510 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.139496    2564 scope.go:117] "RemoveContainer" containerID="01ec6f8e43df226944dde8f9bbba302e041f6d1fa534b9ee00a6c267aef310bb"
Sep 04 23:50:56.139808 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:50:56.139785    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01ec6f8e43df226944dde8f9bbba302e041f6d1fa534b9ee00a6c267aef310bb\": container with ID starting with 01ec6f8e43df226944dde8f9bbba302e041f6d1fa534b9ee00a6c267aef310bb not found: ID does not exist" containerID="01ec6f8e43df226944dde8f9bbba302e041f6d1fa534b9ee00a6c267aef310bb"
Sep 04 23:50:56.139882 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.139815    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01ec6f8e43df226944dde8f9bbba302e041f6d1fa534b9ee00a6c267aef310bb"} err="failed to get container status \"01ec6f8e43df226944dde8f9bbba302e041f6d1fa534b9ee00a6c267aef310bb\": rpc error: code = NotFound desc = could not find container \"01ec6f8e43df226944dde8f9bbba302e041f6d1fa534b9ee00a6c267aef310bb\": container with ID starting with 01ec6f8e43df226944dde8f9bbba302e041f6d1fa534b9ee00a6c267aef310bb not found: ID does not exist"
Sep 04 23:50:56.139882 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.139831    2564 scope.go:117] "RemoveContainer" containerID="dc48442f9de64a3bbd903046cb706be0ae4981e0752b4eea3a39c27799056e99"
Sep 04 23:50:56.153715 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.153694    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"]
Sep 04 23:50:56.157082 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.157061    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-3c6688-76957d48c7-xj9l8"]
Sep 04 23:50:56.175696 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.175673    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d19c2acb-8c60-4edc-a8fb-8d953b007300" path="/var/lib/kubelet/pods/d19c2acb-8c60-4edc-a8fb-8d953b007300/volumes"
Sep 04 23:50:56.175944 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.175929    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"]
Sep 04 23:50:56.179035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:56.179015    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-58fce2d8-5d8ffd4cf8-ppsnv"]
Sep 04 23:50:57.683577 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:57.683544    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:50:58.174882 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:58.174842    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58a3cae7-ea93-49f8-b5aa-6c78a779cc96" path="/var/lib/kubelet/pods/58a3cae7-ea93-49f8-b5aa-6c78a779cc96/volumes"
Sep 04 23:50:59.058440 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.058401    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69" podUID="ad5db71a-d046-40b1-bbc5-48b6333ef4dd" containerName="main" containerID="cri-o://10ae65f48fad2eecdc6dc6807ca2b952b88465019af8fe4922ed6b8cbb71a5a8" gracePeriod=45
Sep 04 23:50:59.314150 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.314099    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:50:59.434706 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.434680    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-home\") pod \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") "
Sep 04 23:50:59.434853 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.434710    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-dshm\") pod \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") "
Sep 04 23:50:59.434853 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.434732    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-model-cache\") pod \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") "
Sep 04 23:50:59.434853 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.434758    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-tmp-dir\") pod \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") "
Sep 04 23:50:59.434853 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.434775    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-tls-certs\") pod \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") "
Sep 04 23:50:59.434853 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.434829    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7kgq\" (UniqueName: \"kubernetes.io/projected/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-kube-api-access-x7kgq\") pod \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\" (UID: \"ad5db71a-d046-40b1-bbc5-48b6333ef4dd\") "
Sep 04 23:50:59.435136 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.434985    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-model-cache" (OuterVolumeSpecName: "model-cache") pod "ad5db71a-d046-40b1-bbc5-48b6333ef4dd" (UID: "ad5db71a-d046-40b1-bbc5-48b6333ef4dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:50:59.435136 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.435006    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-home" (OuterVolumeSpecName: "home") pod "ad5db71a-d046-40b1-bbc5-48b6333ef4dd" (UID: "ad5db71a-d046-40b1-bbc5-48b6333ef4dd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:50:59.435136 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.435022    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ad5db71a-d046-40b1-bbc5-48b6333ef4dd" (UID: "ad5db71a-d046-40b1-bbc5-48b6333ef4dd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:50:59.435136 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.435120    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-tmp-dir\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:50:59.435136 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.435133    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-home\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:50:59.435302 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.435141    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-model-cache\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:50:59.436969 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.436940    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-dshm" (OuterVolumeSpecName: "dshm") pod "ad5db71a-d046-40b1-bbc5-48b6333ef4dd" (UID: "ad5db71a-d046-40b1-bbc5-48b6333ef4dd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:50:59.437082 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.437029    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ad5db71a-d046-40b1-bbc5-48b6333ef4dd" (UID: "ad5db71a-d046-40b1-bbc5-48b6333ef4dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:50:59.437082 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.437053    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-kube-api-access-x7kgq" (OuterVolumeSpecName: "kube-api-access-x7kgq") pod "ad5db71a-d046-40b1-bbc5-48b6333ef4dd" (UID: "ad5db71a-d046-40b1-bbc5-48b6333ef4dd"). InnerVolumeSpecName "kube-api-access-x7kgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:50:59.535544 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.535521    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-dshm\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:50:59.535657 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.535546    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-tls-certs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:50:59.535657 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:50:59.535560    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7kgq\" (UniqueName: \"kubernetes.io/projected/ad5db71a-d046-40b1-bbc5-48b6333ef4dd-kube-api-access-x7kgq\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:51:00.152423 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:00.152395    2564 generic.go:356] "Generic (PLEG): container finished" podID="ad5db71a-d046-40b1-bbc5-48b6333ef4dd" containerID="10ae65f48fad2eecdc6dc6807ca2b952b88465019af8fe4922ed6b8cbb71a5a8" exitCode=0
Sep 04 23:51:00.152833 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:00.152448    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"
Sep 04 23:51:00.152833 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:00.152482    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69" event={"ID":"ad5db71a-d046-40b1-bbc5-48b6333ef4dd","Type":"ContainerDied","Data":"10ae65f48fad2eecdc6dc6807ca2b952b88465019af8fe4922ed6b8cbb71a5a8"}
Sep 04 23:51:00.152833 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:00.152520    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69" event={"ID":"ad5db71a-d046-40b1-bbc5-48b6333ef4dd","Type":"ContainerDied","Data":"14705afed70db7d2a58cf567bce141b7cc151af293b3c5f3a63f3d04f0a2abcc"}
Sep 04 23:51:00.152833 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:00.152537    2564 scope.go:117] "RemoveContainer" containerID="10ae65f48fad2eecdc6dc6807ca2b952b88465019af8fe4922ed6b8cbb71a5a8"
Sep 04 23:51:00.162041 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:00.161999    2564 scope.go:117] "RemoveContainer" containerID="10ae65f48fad2eecdc6dc6807ca2b952b88465019af8fe4922ed6b8cbb71a5a8"
Sep 04 23:51:00.162317 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:51:00.162296    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10ae65f48fad2eecdc6dc6807ca2b952b88465019af8fe4922ed6b8cbb71a5a8\": container with ID starting with 10ae65f48fad2eecdc6dc6807ca2b952b88465019af8fe4922ed6b8cbb71a5a8 not found: ID does not exist" containerID="10ae65f48fad2eecdc6dc6807ca2b952b88465019af8fe4922ed6b8cbb71a5a8"
Sep 04 23:51:00.162394 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:00.162328    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10ae65f48fad2eecdc6dc6807ca2b952b88465019af8fe4922ed6b8cbb71a5a8"} err="failed to get container status \"10ae65f48fad2eecdc6dc6807ca2b952b88465019af8fe4922ed6b8cbb71a5a8\": rpc error: code = NotFound desc = could not find container \"10ae65f48fad2eecdc6dc6807ca2b952b88465019af8fe4922ed6b8cbb71a5a8\": container with ID starting with 10ae65f48fad2eecdc6dc6807ca2b952b88465019af8fe4922ed6b8cbb71a5a8 not found: ID does not exist"
Sep 04 23:51:00.175780 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:00.175757    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"]
Sep 04 23:51:00.179153 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:00.179135    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-3c6688/autores-model-3c6688-kserve-557cf8fb75-wnm69"]
Sep 04 23:51:01.497919 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:01.497866    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:01.680544 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:01.680507    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:01.748706 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:01.748614    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:02.175354 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:02.175324    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad5db71a-d046-40b1-bbc5-48b6333ef4dd" path="/var/lib/kubelet/pods/ad5db71a-d046-40b1-bbc5-48b6333ef4dd/volumes"
Sep 04 23:51:02.304191 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:02.304126    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:02.505477 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:02.505401    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:02.607083 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:02.607050    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:03.631871 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:03.631836    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:03.733061 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:03.732994    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:03.831606 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:03.831578    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:03.932070 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:03.932006    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:04.288086 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:04.288004    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:05.388475 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:05.388442    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:05.794260 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:05.794171    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:06.547425 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:06.547391    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:06.647886 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:06.647854    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:06.970790 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:06.970750    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:07.071496 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:07.071460    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:07.413523 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:07.413491    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:27.619439 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.619408    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"]
Sep 04 23:51:27.620035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.619774    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d19c2acb-8c60-4edc-a8fb-8d953b007300" containerName="payload-processing"
Sep 04 23:51:27.620035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.619786    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19c2acb-8c60-4edc-a8fb-8d953b007300" containerName="payload-processing"
Sep 04 23:51:27.620035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.619806    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58a3cae7-ea93-49f8-b5aa-6c78a779cc96" containerName="payload-pre-processing"
Sep 04 23:51:27.620035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.619811    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="58a3cae7-ea93-49f8-b5aa-6c78a779cc96" containerName="payload-pre-processing"
Sep 04 23:51:27.620035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.619820    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad5db71a-d046-40b1-bbc5-48b6333ef4dd" containerName="main"
Sep 04 23:51:27.620035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.619826    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad5db71a-d046-40b1-bbc5-48b6333ef4dd" containerName="main"
Sep 04 23:51:27.620035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.619880    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="58a3cae7-ea93-49f8-b5aa-6c78a779cc96" containerName="payload-pre-processing"
Sep 04 23:51:27.620035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.619887    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d19c2acb-8c60-4edc-a8fb-8d953b007300" containerName="payload-processing"
Sep 04 23:51:27.620035 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.619923    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad5db71a-d046-40b1-bbc5-48b6333ef4dd" containerName="main"
Sep 04 23:51:27.622931 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.622891    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.625529 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.625512    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-724e1f\"/\"openshift-service-ca.crt\""
Sep 04 23:51:27.626826 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.626801    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-724e1f\"/\"kube-root-ca.crt\""
Sep 04 23:51:27.626826 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.626816    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-724e1f\"/\"test-model-724e1f-kserve-self-signed-certs\""
Sep 04 23:51:27.627043 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.626801    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-724e1f\"/\"default-dockercfg-crf8c\""
Sep 04 23:51:27.632582 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.632533    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"]
Sep 04 23:51:27.658000 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.657955    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:27.781188 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.781145    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-dshm\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.781188 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.781183    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea3d064a-d7d6-4353-aea7-6867eacb4b29-tls-certs\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.781434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.781207    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjljt\" (UniqueName: \"kubernetes.io/projected/ea3d064a-d7d6-4353-aea7-6867eacb4b29-kube-api-access-hjljt\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.781434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.781295    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-home\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.781434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.781353    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-model-cache\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.781434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.781372    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-tmp-dir\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.882699 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.882620    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-model-cache\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.882699 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.882668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-tmp-dir\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.882953 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.882752    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-dshm\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.882953 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.882780    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea3d064a-d7d6-4353-aea7-6867eacb4b29-tls-certs\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.882953 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.882817    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjljt\" (UniqueName: \"kubernetes.io/projected/ea3d064a-d7d6-4353-aea7-6867eacb4b29-kube-api-access-hjljt\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.882953 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.882858    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-home\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.883163 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.883081    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-model-cache\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.883163 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.883147    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-tmp-dir\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.883278 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.883225    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-home\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.885124 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.885097    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-dshm\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.885378 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.885359    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea3d064a-d7d6-4353-aea7-6867eacb4b29-tls-certs\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.891005 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.890980    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjljt\" (UniqueName: \"kubernetes.io/projected/ea3d064a-d7d6-4353-aea7-6867eacb4b29-kube-api-access-hjljt\") pod \"test-model-724e1f-kserve-d764f4947-f65n7\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") " pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:27.932953 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:27.932923    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:28.203289 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:28.203256    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"]
Sep 04 23:51:28.205721 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:51:28.205678    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea3d064a_d7d6_4353_aea7_6867eacb4b29.slice/crio-b40cd5d2de20454546e13e9ae6cbb3375a9fbfc0e9a78cdf3b045ff65875072d WatchSource:0}: Error finding container b40cd5d2de20454546e13e9ae6cbb3375a9fbfc0e9a78cdf3b045ff65875072d: Status 404 returned error can't find the container with id b40cd5d2de20454546e13e9ae6cbb3375a9fbfc0e9a78cdf3b045ff65875072d
Sep 04 23:51:28.293542 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:28.293515    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7" event={"ID":"ea3d064a-d7d6-4353-aea7-6867eacb4b29","Type":"ContainerStarted","Data":"5529125c402d9945c73f2d06264dff4666617d556cac2b107a46450631ffd983"}
Sep 04 23:51:28.293696 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:28.293563    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7" event={"ID":"ea3d064a-d7d6-4353-aea7-6867eacb4b29","Type":"ContainerStarted","Data":"b40cd5d2de20454546e13e9ae6cbb3375a9fbfc0e9a78cdf3b045ff65875072d"}
Sep 04 23:51:28.309564 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:28.309519    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7" podStartSLOduration=1.3095058609999999 podStartE2EDuration="1.309505861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:51:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:51:28.307543219 +0000 UTC m=+876.555543754" watchObservedRunningTime="2026-09-04 23:51:28.309505861 +0000 UTC m=+876.557506376"
Sep 04 23:51:30.253867 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:30.253826    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:30.354879 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:30.354840    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:36.309632 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:36.309595    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:36.377348 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:36.377308    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:37.933610 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:37.933578    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:37.934003 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:37.933621    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:37.945998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:37.945976    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:38.348640 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:38.348565    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:51:51.747679 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:51.747595    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:56.841998 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:56.841963    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:57.703327 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:57.703279    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:57.804324 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:57.804277    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:51:57.903764 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:51:57.903712    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:02.887455 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:02.887419    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:08.361767 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:08.361713    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:09.161606 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:09.161554    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:13.426008 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:13.425967    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:13.813160 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:13.813077    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:13.915242 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:13.915207    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:14.014692 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:14.014652    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:14.701300 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:14.701264    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:14.803264 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:14.803231    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:15.103790 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:15.103696    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:15.212804 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:15.212762    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:15.769536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:15.769508    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:15.867607 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:15.867575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:17.332867 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:17.332824    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:19.838530 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:19.838484    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:19.937248 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:19.937214    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:20.050327 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:20.050295    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:20.445614 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:20.445584    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:20.787264 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:20.787174    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:20.892601 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:20.892563    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:22.574020 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:22.573978    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:22.803347 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:22.803310    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:22.904093 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:22.904057    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:23.760752 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:23.760715    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:23.862547 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:23.862512    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:24.425799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:24.425760    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:30.200519 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:30.200480    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:30.301367 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:30.301338    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:30.430963 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:30.430926    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:30.726495 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:30.726464    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:30.833214 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:30.833181    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:32.105188 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:32.105151    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:33.912075 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:33.912032    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:34.012800 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:34.012763    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:35.203222 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:35.203188    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:35.303155 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:35.303120    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:35.353996 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:35.353962    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:35.622628 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:35.622592    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:35.718150 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:35.718119    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:35.794553 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:35.794523    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:37.320214 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:37.320170    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:37.394735 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:37.394702    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"]
Sep 04 23:52:37.397626 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:37.397595    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:37.448272 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:37.448241    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:40.197013 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:40.196969    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"]
Sep 04 23:52:40.217109 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:40.217078    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:40.414970 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:40.414938    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:41.083451 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:41.083416    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:50.755645 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:50.755608    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:50.779003 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:50.778970    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"]
Sep 04 23:52:50.779244 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:50.779201    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf" podUID="2b8976c5-6336-4a19-8749-5d4d966178c9" containerName="payload-processing" containerID="cri-o://3f247f6335c0bf3cbf7d75efcefee001a37808bf58e17686ce73df7929908fbb" gracePeriod=30
Sep 04 23:52:50.785216 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:50.785187    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"]
Sep 04 23:52:50.785459 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:50.785417    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc" podUID="6c87c053-a412-48f7-9981-849412a38263" containerName="payload-pre-processing" containerID="cri-o://a8655dece554dcd20966d640d96732a88f14bacbd43fbb0b39d13108c27905d5" gracePeriod=30
Sep 04 23:52:50.819540 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:50.819512    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:50.879483 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:50.879442    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:50.948152 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:50.948125    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:51.010175 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.010143    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:51.032939 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.032918    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"
Sep 04 23:52:51.062658 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.062636    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"
Sep 04 23:52:51.112346 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.112320    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr6m7\" (UniqueName: \"kubernetes.io/projected/6c87c053-a412-48f7-9981-849412a38263-kube-api-access-jr6m7\") pod \"6c87c053-a412-48f7-9981-849412a38263\" (UID: \"6c87c053-a412-48f7-9981-849412a38263\") "
Sep 04 23:52:51.112483 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.112348    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88sv9\" (UniqueName: \"kubernetes.io/projected/2b8976c5-6336-4a19-8749-5d4d966178c9-kube-api-access-88sv9\") pod \"2b8976c5-6336-4a19-8749-5d4d966178c9\" (UID: \"2b8976c5-6336-4a19-8749-5d4d966178c9\") "
Sep 04 23:52:51.112483 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.112468    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b8976c5-6336-4a19-8749-5d4d966178c9-plugins-config-volume\") pod \"2b8976c5-6336-4a19-8749-5d4d966178c9\" (UID: \"2b8976c5-6336-4a19-8749-5d4d966178c9\") "
Sep 04 23:52:51.112604 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.112491    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c87c053-a412-48f7-9981-849412a38263-plugins-config-volume\") pod \"6c87c053-a412-48f7-9981-849412a38263\" (UID: \"6c87c053-a412-48f7-9981-849412a38263\") "
Sep 04 23:52:51.112864 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.112838    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b8976c5-6336-4a19-8749-5d4d966178c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2b8976c5-6336-4a19-8749-5d4d966178c9" (UID: "2b8976c5-6336-4a19-8749-5d4d966178c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:52:51.113067 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.112877    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c87c053-a412-48f7-9981-849412a38263-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6c87c053-a412-48f7-9981-849412a38263" (UID: "6c87c053-a412-48f7-9981-849412a38263"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:52:51.114515 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.114495    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c87c053-a412-48f7-9981-849412a38263-kube-api-access-jr6m7" (OuterVolumeSpecName: "kube-api-access-jr6m7") pod "6c87c053-a412-48f7-9981-849412a38263" (UID: "6c87c053-a412-48f7-9981-849412a38263"). InnerVolumeSpecName "kube-api-access-jr6m7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:52:51.114599 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.114532    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b8976c5-6336-4a19-8749-5d4d966178c9-kube-api-access-88sv9" (OuterVolumeSpecName: "kube-api-access-88sv9") pod "2b8976c5-6336-4a19-8749-5d4d966178c9" (UID: "2b8976c5-6336-4a19-8749-5d4d966178c9"). InnerVolumeSpecName "kube-api-access-88sv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:52:51.213376 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.213349    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b8976c5-6336-4a19-8749-5d4d966178c9-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:51.213376 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.213373    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c87c053-a412-48f7-9981-849412a38263-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:51.213531 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.213386    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr6m7\" (UniqueName: \"kubernetes.io/projected/6c87c053-a412-48f7-9981-849412a38263-kube-api-access-jr6m7\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:51.213531 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.213395    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88sv9\" (UniqueName: \"kubernetes.io/projected/2b8976c5-6336-4a19-8749-5d4d966178c9-kube-api-access-88sv9\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:51.281390 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.281331    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:51.699285 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.699250    2564 generic.go:356] "Generic (PLEG): container finished" podID="2b8976c5-6336-4a19-8749-5d4d966178c9" containerID="3f247f6335c0bf3cbf7d75efcefee001a37808bf58e17686ce73df7929908fbb" exitCode=0
Sep 04 23:52:51.699438 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.699290    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"
Sep 04 23:52:51.699438 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.699339    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf" event={"ID":"2b8976c5-6336-4a19-8749-5d4d966178c9","Type":"ContainerDied","Data":"3f247f6335c0bf3cbf7d75efcefee001a37808bf58e17686ce73df7929908fbb"}
Sep 04 23:52:51.699438 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.699395    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf" event={"ID":"2b8976c5-6336-4a19-8749-5d4d966178c9","Type":"ContainerDied","Data":"8046700316a1d71d8d2f236a711c03274dc1798bd502c46f1fc6860f55f95c9e"}
Sep 04 23:52:51.699438 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.699428    2564 scope.go:117] "RemoveContainer" containerID="3f247f6335c0bf3cbf7d75efcefee001a37808bf58e17686ce73df7929908fbb"
Sep 04 23:52:51.700780 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.700761    2564 generic.go:356] "Generic (PLEG): container finished" podID="6c87c053-a412-48f7-9981-849412a38263" containerID="a8655dece554dcd20966d640d96732a88f14bacbd43fbb0b39d13108c27905d5" exitCode=0
Sep 04 23:52:51.700862 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.700801    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"
Sep 04 23:52:51.700935 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.700852    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc" event={"ID":"6c87c053-a412-48f7-9981-849412a38263","Type":"ContainerDied","Data":"a8655dece554dcd20966d640d96732a88f14bacbd43fbb0b39d13108c27905d5"}
Sep 04 23:52:51.701006 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.700944    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc" event={"ID":"6c87c053-a412-48f7-9981-849412a38263","Type":"ContainerDied","Data":"766411719d87de8430bc9c06543a38198348dbcf691aba0da11aa5df0b6bbb23"}
Sep 04 23:52:51.709811 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.709792    2564 scope.go:117] "RemoveContainer" containerID="3f247f6335c0bf3cbf7d75efcefee001a37808bf58e17686ce73df7929908fbb"
Sep 04 23:52:51.710122 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:52:51.710100    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f247f6335c0bf3cbf7d75efcefee001a37808bf58e17686ce73df7929908fbb\": container with ID starting with 3f247f6335c0bf3cbf7d75efcefee001a37808bf58e17686ce73df7929908fbb not found: ID does not exist" containerID="3f247f6335c0bf3cbf7d75efcefee001a37808bf58e17686ce73df7929908fbb"
Sep 04 23:52:51.710217 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.710131    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f247f6335c0bf3cbf7d75efcefee001a37808bf58e17686ce73df7929908fbb"} err="failed to get container status \"3f247f6335c0bf3cbf7d75efcefee001a37808bf58e17686ce73df7929908fbb\": rpc error: code = NotFound desc = could not find container \"3f247f6335c0bf3cbf7d75efcefee001a37808bf58e17686ce73df7929908fbb\": container with ID starting with 3f247f6335c0bf3cbf7d75efcefee001a37808bf58e17686ce73df7929908fbb not found: ID does not exist"
Sep 04 23:52:51.710217 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.710146    2564 scope.go:117] "RemoveContainer" containerID="a8655dece554dcd20966d640d96732a88f14bacbd43fbb0b39d13108c27905d5"
Sep 04 23:52:51.718263 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.718247    2564 scope.go:117] "RemoveContainer" containerID="a8655dece554dcd20966d640d96732a88f14bacbd43fbb0b39d13108c27905d5"
Sep 04 23:52:51.718538 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:52:51.718520    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8655dece554dcd20966d640d96732a88f14bacbd43fbb0b39d13108c27905d5\": container with ID starting with a8655dece554dcd20966d640d96732a88f14bacbd43fbb0b39d13108c27905d5 not found: ID does not exist" containerID="a8655dece554dcd20966d640d96732a88f14bacbd43fbb0b39d13108c27905d5"
Sep 04 23:52:51.718586 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.718545    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8655dece554dcd20966d640d96732a88f14bacbd43fbb0b39d13108c27905d5"} err="failed to get container status \"a8655dece554dcd20966d640d96732a88f14bacbd43fbb0b39d13108c27905d5\": rpc error: code = NotFound desc = could not find container \"a8655dece554dcd20966d640d96732a88f14bacbd43fbb0b39d13108c27905d5\": container with ID starting with a8655dece554dcd20966d640d96732a88f14bacbd43fbb0b39d13108c27905d5 not found: ID does not exist"
Sep 04 23:52:51.722248 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.722230    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"]
Sep 04 23:52:51.726009 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.725989    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d8b443-6c5559fcbc-drngf"]
Sep 04 23:52:51.735915 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.735885    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"]
Sep 04 23:52:51.739392 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:51.739364    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d8b443-6fc54688db-jd8bc"]
Sep 04 23:52:52.175438 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.175410    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b8976c5-6336-4a19-8749-5d4d966178c9" path="/var/lib/kubelet/pods/2b8976c5-6336-4a19-8749-5d4d966178c9/volumes"
Sep 04 23:52:52.175944 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.175924    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c87c053-a412-48f7-9981-849412a38263" path="/var/lib/kubelet/pods/6c87c053-a412-48f7-9981-849412a38263/volumes"
Sep 04 23:52:52.416204 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.416169    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p" podUID="faf89f99-d54b-4757-a109-4d752bea81db" containerName="main" containerID="cri-o://3a900991933dc7669ed12e4afe2d684ff3fc62231955ce6c578b77556503e00c" gracePeriod=45
Sep 04 23:52:52.661399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.661378    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:52:52.706661 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.706585    2564 generic.go:356] "Generic (PLEG): container finished" podID="faf89f99-d54b-4757-a109-4d752bea81db" containerID="3a900991933dc7669ed12e4afe2d684ff3fc62231955ce6c578b77556503e00c" exitCode=0
Sep 04 23:52:52.706661 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.706641    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"
Sep 04 23:52:52.706661 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.706650    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p" event={"ID":"faf89f99-d54b-4757-a109-4d752bea81db","Type":"ContainerDied","Data":"3a900991933dc7669ed12e4afe2d684ff3fc62231955ce6c578b77556503e00c"}
Sep 04 23:52:52.707025 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.706681    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p" event={"ID":"faf89f99-d54b-4757-a109-4d752bea81db","Type":"ContainerDied","Data":"fddba322fe11df8f57f795032f5196cac7db0756eba8c596c281dc64003ef39f"}
Sep 04 23:52:52.707025 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.706700    2564 scope.go:117] "RemoveContainer" containerID="3a900991933dc7669ed12e4afe2d684ff3fc62231955ce6c578b77556503e00c"
Sep 04 23:52:52.716434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.716416    2564 scope.go:117] "RemoveContainer" containerID="3a900991933dc7669ed12e4afe2d684ff3fc62231955ce6c578b77556503e00c"
Sep 04 23:52:52.716779 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:52:52.716758    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a900991933dc7669ed12e4afe2d684ff3fc62231955ce6c578b77556503e00c\": container with ID starting with 3a900991933dc7669ed12e4afe2d684ff3fc62231955ce6c578b77556503e00c not found: ID does not exist" containerID="3a900991933dc7669ed12e4afe2d684ff3fc62231955ce6c578b77556503e00c"
Sep 04 23:52:52.716924 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.716786    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a900991933dc7669ed12e4afe2d684ff3fc62231955ce6c578b77556503e00c"} err="failed to get container status \"3a900991933dc7669ed12e4afe2d684ff3fc62231955ce6c578b77556503e00c\": rpc error: code = NotFound desc = could not find container \"3a900991933dc7669ed12e4afe2d684ff3fc62231955ce6c578b77556503e00c\": container with ID starting with 3a900991933dc7669ed12e4afe2d684ff3fc62231955ce6c578b77556503e00c not found: ID does not exist"
Sep 04 23:52:52.727075 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.727057    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgqmk\" (UniqueName: \"kubernetes.io/projected/faf89f99-d54b-4757-a109-4d752bea81db-kube-api-access-wgqmk\") pod \"faf89f99-d54b-4757-a109-4d752bea81db\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") "
Sep 04 23:52:52.727156 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.727090    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-tmp-dir\") pod \"faf89f99-d54b-4757-a109-4d752bea81db\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") "
Sep 04 23:52:52.727227 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.727211    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-model-cache\") pod \"faf89f99-d54b-4757-a109-4d752bea81db\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") "
Sep 04 23:52:52.727276 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.727241    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-dshm\") pod \"faf89f99-d54b-4757-a109-4d752bea81db\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") "
Sep 04 23:52:52.727326 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.727282    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faf89f99-d54b-4757-a109-4d752bea81db-tls-certs\") pod \"faf89f99-d54b-4757-a109-4d752bea81db\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") "
Sep 04 23:52:52.727382 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.727324    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "faf89f99-d54b-4757-a109-4d752bea81db" (UID: "faf89f99-d54b-4757-a109-4d752bea81db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:52:52.727382 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.727335    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-home\") pod \"faf89f99-d54b-4757-a109-4d752bea81db\" (UID: \"faf89f99-d54b-4757-a109-4d752bea81db\") "
Sep 04 23:52:52.727531 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.727503    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-model-cache" (OuterVolumeSpecName: "model-cache") pod "faf89f99-d54b-4757-a109-4d752bea81db" (UID: "faf89f99-d54b-4757-a109-4d752bea81db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:52:52.727607 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.727534    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-home" (OuterVolumeSpecName: "home") pod "faf89f99-d54b-4757-a109-4d752bea81db" (UID: "faf89f99-d54b-4757-a109-4d752bea81db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:52:52.727733 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.727714    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-model-cache\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:52.727909 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.727736    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-home\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:52.727909 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.727749    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-tmp-dir\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:52.729317 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.729290    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faf89f99-d54b-4757-a109-4d752bea81db-kube-api-access-wgqmk" (OuterVolumeSpecName: "kube-api-access-wgqmk") pod "faf89f99-d54b-4757-a109-4d752bea81db" (UID: "faf89f99-d54b-4757-a109-4d752bea81db"). InnerVolumeSpecName "kube-api-access-wgqmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:52:52.729402 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.729328    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-dshm" (OuterVolumeSpecName: "dshm") pod "faf89f99-d54b-4757-a109-4d752bea81db" (UID: "faf89f99-d54b-4757-a109-4d752bea81db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:52:52.729445 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.729423    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faf89f99-d54b-4757-a109-4d752bea81db-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "faf89f99-d54b-4757-a109-4d752bea81db" (UID: "faf89f99-d54b-4757-a109-4d752bea81db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:52:52.828187 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.828161    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgqmk\" (UniqueName: \"kubernetes.io/projected/faf89f99-d54b-4757-a109-4d752bea81db-kube-api-access-wgqmk\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:52.828187 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.828186    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/faf89f99-d54b-4757-a109-4d752bea81db-dshm\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:52.828309 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:52.828202    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faf89f99-d54b-4757-a109-4d752bea81db-tls-certs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:53.028615 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:53.028591    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"]
Sep 04 23:52:53.034215 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:53.034189    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-d8b443/test-model-d8b443-kserve-748bdd646c-gvp5p"]
Sep 04 23:52:54.174295 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:54.174260    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faf89f99-d54b-4757-a109-4d752bea81db" path="/var/lib/kubelet/pods/faf89f99-d54b-4757-a109-4d752bea81db/volumes"
Sep 04 23:52:55.217330 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.217272    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7" podUID="ea3d064a-d7d6-4353-aea7-6867eacb4b29" containerName="main" containerID="cri-o://5529125c402d9945c73f2d06264dff4666617d556cac2b107a46450631ffd983" gracePeriod=45
Sep 04 23:52:55.471922 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.471854    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:52:55.552234 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.552208    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-model-cache\") pod \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") "
Sep 04 23:52:55.552350 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.552240    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-home\") pod \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") "
Sep 04 23:52:55.552350 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.552273    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea3d064a-d7d6-4353-aea7-6867eacb4b29-tls-certs\") pod \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") "
Sep 04 23:52:55.552350 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.552329    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-dshm\") pod \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") "
Sep 04 23:52:55.552465 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.552356    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjljt\" (UniqueName: \"kubernetes.io/projected/ea3d064a-d7d6-4353-aea7-6867eacb4b29-kube-api-access-hjljt\") pod \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") "
Sep 04 23:52:55.552465 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.552389    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-tmp-dir\") pod \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\" (UID: \"ea3d064a-d7d6-4353-aea7-6867eacb4b29\") "
Sep 04 23:52:55.552544 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.552493    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-model-cache" (OuterVolumeSpecName: "model-cache") pod "ea3d064a-d7d6-4353-aea7-6867eacb4b29" (UID: "ea3d064a-d7d6-4353-aea7-6867eacb4b29"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:52:55.552544 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.552501    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-home" (OuterVolumeSpecName: "home") pod "ea3d064a-d7d6-4353-aea7-6867eacb4b29" (UID: "ea3d064a-d7d6-4353-aea7-6867eacb4b29"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:52:55.552640 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.552614    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-model-cache\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:55.552680 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.552659    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-home\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:55.552734 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.552700    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ea3d064a-d7d6-4353-aea7-6867eacb4b29" (UID: "ea3d064a-d7d6-4353-aea7-6867eacb4b29"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:52:55.554635 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.554598    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-dshm" (OuterVolumeSpecName: "dshm") pod "ea3d064a-d7d6-4353-aea7-6867eacb4b29" (UID: "ea3d064a-d7d6-4353-aea7-6867eacb4b29"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:52:55.554733 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.554672    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea3d064a-d7d6-4353-aea7-6867eacb4b29-kube-api-access-hjljt" (OuterVolumeSpecName: "kube-api-access-hjljt") pod "ea3d064a-d7d6-4353-aea7-6867eacb4b29" (UID: "ea3d064a-d7d6-4353-aea7-6867eacb4b29"). InnerVolumeSpecName "kube-api-access-hjljt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:52:55.554733 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.554671    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea3d064a-d7d6-4353-aea7-6867eacb4b29-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ea3d064a-d7d6-4353-aea7-6867eacb4b29" (UID: "ea3d064a-d7d6-4353-aea7-6867eacb4b29"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:52:55.653533 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.653501    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-dshm\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:55.653533 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.653525    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjljt\" (UniqueName: \"kubernetes.io/projected/ea3d064a-d7d6-4353-aea7-6867eacb4b29-kube-api-access-hjljt\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:55.653698 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.653540    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea3d064a-d7d6-4353-aea7-6867eacb4b29-tmp-dir\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:55.653698 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.653554    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea3d064a-d7d6-4353-aea7-6867eacb4b29-tls-certs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:52:55.722677 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.722624    2564 generic.go:356] "Generic (PLEG): container finished" podID="ea3d064a-d7d6-4353-aea7-6867eacb4b29" containerID="5529125c402d9945c73f2d06264dff4666617d556cac2b107a46450631ffd983" exitCode=0
Sep 04 23:52:55.722677 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.722666    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"
Sep 04 23:52:55.722817 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.722710    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7" event={"ID":"ea3d064a-d7d6-4353-aea7-6867eacb4b29","Type":"ContainerDied","Data":"5529125c402d9945c73f2d06264dff4666617d556cac2b107a46450631ffd983"}
Sep 04 23:52:55.722817 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.722747    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7" event={"ID":"ea3d064a-d7d6-4353-aea7-6867eacb4b29","Type":"ContainerDied","Data":"b40cd5d2de20454546e13e9ae6cbb3375a9fbfc0e9a78cdf3b045ff65875072d"}
Sep 04 23:52:55.722817 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.722766    2564 scope.go:117] "RemoveContainer" containerID="5529125c402d9945c73f2d06264dff4666617d556cac2b107a46450631ffd983"
Sep 04 23:52:55.731580 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.731566    2564 scope.go:117] "RemoveContainer" containerID="5529125c402d9945c73f2d06264dff4666617d556cac2b107a46450631ffd983"
Sep 04 23:52:55.731837 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:52:55.731820    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5529125c402d9945c73f2d06264dff4666617d556cac2b107a46450631ffd983\": container with ID starting with 5529125c402d9945c73f2d06264dff4666617d556cac2b107a46450631ffd983 not found: ID does not exist" containerID="5529125c402d9945c73f2d06264dff4666617d556cac2b107a46450631ffd983"
Sep 04 23:52:55.731873 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.731847    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5529125c402d9945c73f2d06264dff4666617d556cac2b107a46450631ffd983"} err="failed to get container status \"5529125c402d9945c73f2d06264dff4666617d556cac2b107a46450631ffd983\": rpc error: code = NotFound desc = could not find container \"5529125c402d9945c73f2d06264dff4666617d556cac2b107a46450631ffd983\": container with ID starting with 5529125c402d9945c73f2d06264dff4666617d556cac2b107a46450631ffd983 not found: ID does not exist"
Sep 04 23:52:55.743322 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.743302    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"]
Sep 04 23:52:55.745653 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:55.745629    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-724e1f/test-model-724e1f-kserve-d764f4947-f65n7"]
Sep 04 23:52:56.173887 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:56.173859    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea3d064a-d7d6-4353-aea7-6867eacb4b29" path="/var/lib/kubelet/pods/ea3d064a-d7d6-4353-aea7-6867eacb4b29/volumes"
Sep 04 23:52:56.894412 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:56.894376    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:57.770131 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.770096    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx"]
Sep 04 23:52:57.770471 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.770458    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea3d064a-d7d6-4353-aea7-6867eacb4b29" containerName="main"
Sep 04 23:52:57.770519 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.770474    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea3d064a-d7d6-4353-aea7-6867eacb4b29" containerName="main"
Sep 04 23:52:57.770519 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.770495    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faf89f99-d54b-4757-a109-4d752bea81db" containerName="main"
Sep 04 23:52:57.770519 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.770501    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="faf89f99-d54b-4757-a109-4d752bea81db" containerName="main"
Sep 04 23:52:57.770519 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.770510    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c87c053-a412-48f7-9981-849412a38263" containerName="payload-pre-processing"
Sep 04 23:52:57.770519 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.770515    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c87c053-a412-48f7-9981-849412a38263" containerName="payload-pre-processing"
Sep 04 23:52:57.770676 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.770523    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b8976c5-6336-4a19-8749-5d4d966178c9" containerName="payload-processing"
Sep 04 23:52:57.770676 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.770532    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b8976c5-6336-4a19-8749-5d4d966178c9" containerName="payload-processing"
Sep 04 23:52:57.770676 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.770588    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="faf89f99-d54b-4757-a109-4d752bea81db" containerName="main"
Sep 04 23:52:57.770676 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.770597    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea3d064a-d7d6-4353-aea7-6867eacb4b29" containerName="main"
Sep 04 23:52:57.770676 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.770604    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c87c053-a412-48f7-9981-849412a38263" containerName="payload-pre-processing"
Sep 04 23:52:57.770676 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.770617    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b8976c5-6336-4a19-8749-5d4d966178c9" containerName="payload-processing"
Sep 04 23:52:57.774593 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.774577    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx"
Sep 04 23:52:57.777097 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.777076    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 23:52:57.777097 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.777091    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k4f64\""
Sep 04 23:52:57.780614 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.780592    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx"]
Sep 04 23:52:57.870021 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.869993    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-724e1f-758cx\" (UID: \"13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx"
Sep 04 23:52:57.971235 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.971209    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-724e1f-758cx\" (UID: \"13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx"
Sep 04 23:52:57.971760 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:57.971740    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-724e1f-758cx\" (UID: \"13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx"
Sep 04 23:52:58.086220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:58.086165    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx"
Sep 04 23:52:58.229229 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:58.229194    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:58.355449 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:58.355427    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx"]
Sep 04 23:52:58.357740 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:52:58.357714    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13bfc0cc_6bce_49e8_bcd5_7caff7bd8d78.slice/crio-0f57269774659e3f2af75d636906fad6a1bd8db4ee42288810c90413cd8c825a WatchSource:0}: Error finding container 0f57269774659e3f2af75d636906fad6a1bd8db4ee42288810c90413cd8c825a: Status 404 returned error can't find the container with id 0f57269774659e3f2af75d636906fad6a1bd8db4ee42288810c90413cd8c825a
Sep 04 23:52:58.737546 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:58.737512    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx" event={"ID":"13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78","Type":"ContainerStarted","Data":"0f57269774659e3f2af75d636906fad6a1bd8db4ee42288810c90413cd8c825a"}
Sep 04 23:52:59.173709 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:59.173671    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:59.273512 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:59.273477    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:59.457003 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:59.456888    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:52:59.557724 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:52:59.557692    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:00.576550 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:00.576511    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:01.754234 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:01.754146    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx" event={"ID":"13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78","Type":"ContainerStarted","Data":"5b45c3f63fe790d5c980c3f2c6d529e81870141e276b9baacb13415dcdc5202d"}
Sep 04 23:53:01.768796 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:01.768745    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx" podStartSLOduration=2.061024466 podStartE2EDuration="4.768731089s" totalImagesPullingTime="2.707706623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:52:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:53:01.768304205 +0000 UTC m=+970.016304746" watchObservedRunningTime="2026-09-04 23:53:01.768731089 +0000 UTC m=+970.016731605"
Sep 04 23:53:02.760097 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:02.760060    2564 generic.go:356] "Generic (PLEG): container finished" podID="13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78" containerID="5b45c3f63fe790d5c980c3f2c6d529e81870141e276b9baacb13415dcdc5202d" exitCode=0
Sep 04 23:53:02.760486 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:02.760104    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx" event={"ID":"13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78","Type":"ContainerDied","Data":"5b45c3f63fe790d5c980c3f2c6d529e81870141e276b9baacb13415dcdc5202d"}
Sep 04 23:53:03.894367 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:03.894340    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx"
Sep 04 23:53:04.029677 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:04.029590    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78-maas-api-service-ca\") pod \"13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78\" (UID: \"13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78\") "
Sep 04 23:53:04.030016 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:04.029990    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78" (UID: "13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:53:04.131156 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:04.131122    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78-maas-api-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:53:04.771464 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:04.771421    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx" event={"ID":"13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78","Type":"ContainerDied","Data":"0f57269774659e3f2af75d636906fad6a1bd8db4ee42288810c90413cd8c825a"}
Sep 04 23:53:04.771464 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:04.771459    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx"
Sep 04 23:53:04.771734 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:04.771478    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f57269774659e3f2af75d636906fad6a1bd8db4ee42288810c90413cd8c825a"
Sep 04 23:53:05.530366 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:05.530324    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:05.631874 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:05.631839    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:05.855074 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:05.854985    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8"]
Sep 04 23:53:05.855368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:05.855353    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78" containerName="revoke-keys"
Sep 04 23:53:05.855368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:05.855366    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78" containerName="revoke-keys"
Sep 04 23:53:05.855523 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:05.855425    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78" containerName="revoke-keys"
Sep 04 23:53:05.857988 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:05.857944    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8"
Sep 04 23:53:05.860434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:05.860399    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 23:53:05.860556 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:05.860469    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k4f64\""
Sep 04 23:53:05.865030 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:05.865007    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8"]
Sep 04 23:53:05.948156 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:05.948124    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3aa4556-4dd7-4ce6-95db-2a9511fb951e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8\" (UID: \"a3aa4556-4dd7-4ce6-95db-2a9511fb951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8"
Sep 04 23:53:06.049218 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:06.049182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3aa4556-4dd7-4ce6-95db-2a9511fb951e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8\" (UID: \"a3aa4556-4dd7-4ce6-95db-2a9511fb951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8"
Sep 04 23:53:06.049845 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:06.049823    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3aa4556-4dd7-4ce6-95db-2a9511fb951e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8\" (UID: \"a3aa4556-4dd7-4ce6-95db-2a9511fb951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8"
Sep 04 23:53:06.170867 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:06.170828    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8"
Sep 04 23:53:06.437789 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:06.437765    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8"]
Sep 04 23:53:06.439973 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:53:06.439945    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3aa4556_4dd7_4ce6_95db_2a9511fb951e.slice/crio-ee0f45de67ed1d52781294aa844fc3f59ddfb74e78e7eeb36799cba7572fb77c WatchSource:0}: Error finding container ee0f45de67ed1d52781294aa844fc3f59ddfb74e78e7eeb36799cba7572fb77c: Status 404 returned error can't find the container with id ee0f45de67ed1d52781294aa844fc3f59ddfb74e78e7eeb36799cba7572fb77c
Sep 04 23:53:06.782412 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:06.782383    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8" event={"ID":"a3aa4556-4dd7-4ce6-95db-2a9511fb951e","Type":"ContainerStarted","Data":"322ebedae192c3ee498bb21794d2df5127ee3ec0115fcb6884b1a0072b8e702d"}
Sep 04 23:53:06.782808 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:06.782433    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8" event={"ID":"a3aa4556-4dd7-4ce6-95db-2a9511fb951e","Type":"ContainerStarted","Data":"ee0f45de67ed1d52781294aa844fc3f59ddfb74e78e7eeb36799cba7572fb77c"}
Sep 04 23:53:06.798147 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:06.798100    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8" podStartSLOduration=1.7980853749999999 podStartE2EDuration="1.798085375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:53:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:53:06.795225276 +0000 UTC m=+975.043225793" watchObservedRunningTime="2026-09-04 23:53:06.798085375 +0000 UTC m=+975.046085891"
Sep 04 23:53:07.794316 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:07.794280    2564 generic.go:356] "Generic (PLEG): container finished" podID="a3aa4556-4dd7-4ce6-95db-2a9511fb951e" containerID="322ebedae192c3ee498bb21794d2df5127ee3ec0115fcb6884b1a0072b8e702d" exitCode=0
Sep 04 23:53:07.794704 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:07.794361    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8" event={"ID":"a3aa4556-4dd7-4ce6-95db-2a9511fb951e","Type":"ContainerDied","Data":"322ebedae192c3ee498bb21794d2df5127ee3ec0115fcb6884b1a0072b8e702d"}
Sep 04 23:53:07.897394 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:07.897364    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:08.521198 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.521109    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:08.540592 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.540562    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"]
Sep 04 23:53:08.540786 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.540755    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk" podUID="7183cdd7-779c-434a-98e1-d9952bdf6ee9" containerName="payload-processing" containerID="cri-o://c149315714a66abfc22bca92c98226f6ded4f8484e4efa0954210cd8c37bca47" gracePeriod=30
Sep 04 23:53:08.547327 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.547298    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"]
Sep 04 23:53:08.547547 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.547521    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n" podUID="4a0bfc6d-d753-4c24-b760-3afb506c824e" containerName="payload-pre-processing" containerID="cri-o://cf131e49909d32f53a40e37d09cfbd26bb6701d7b662042cafe5b3bd6a9c844a" gracePeriod=30
Sep 04 23:53:08.582430 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.582387    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:08.641744 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.641705    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:08.686442 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.686410    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx"]
Sep 04 23:53:08.689535 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.689505    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-724e1f-758cx"]
Sep 04 23:53:08.701049 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.701018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:08.803788 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.803699    2564 generic.go:356] "Generic (PLEG): container finished" podID="7183cdd7-779c-434a-98e1-d9952bdf6ee9" containerID="c149315714a66abfc22bca92c98226f6ded4f8484e4efa0954210cd8c37bca47" exitCode=0
Sep 04 23:53:08.804238 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.803781    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk" event={"ID":"7183cdd7-779c-434a-98e1-d9952bdf6ee9","Type":"ContainerDied","Data":"c149315714a66abfc22bca92c98226f6ded4f8484e4efa0954210cd8c37bca47"}
Sep 04 23:53:08.805816 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.805782    2564 generic.go:356] "Generic (PLEG): container finished" podID="4a0bfc6d-d753-4c24-b760-3afb506c824e" containerID="cf131e49909d32f53a40e37d09cfbd26bb6701d7b662042cafe5b3bd6a9c844a" exitCode=0
Sep 04 23:53:08.805955 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.805848    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n" event={"ID":"4a0bfc6d-d753-4c24-b760-3afb506c824e","Type":"ContainerDied","Data":"cf131e49909d32f53a40e37d09cfbd26bb6701d7b662042cafe5b3bd6a9c844a"}
Sep 04 23:53:08.828659 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.828636    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"
Sep 04 23:53:08.831475 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.831459    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"
Sep 04 23:53:08.943727 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.943704    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8"
Sep 04 23:53:08.976533 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.976509    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a0bfc6d-d753-4c24-b760-3afb506c824e-plugins-config-volume\") pod \"4a0bfc6d-d753-4c24-b760-3afb506c824e\" (UID: \"4a0bfc6d-d753-4c24-b760-3afb506c824e\") "
Sep 04 23:53:08.976670 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.976586    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7183cdd7-779c-434a-98e1-d9952bdf6ee9-plugins-config-volume\") pod \"7183cdd7-779c-434a-98e1-d9952bdf6ee9\" (UID: \"7183cdd7-779c-434a-98e1-d9952bdf6ee9\") "
Sep 04 23:53:08.976741 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.976669    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c25x2\" (UniqueName: \"kubernetes.io/projected/4a0bfc6d-d753-4c24-b760-3afb506c824e-kube-api-access-c25x2\") pod \"4a0bfc6d-d753-4c24-b760-3afb506c824e\" (UID: \"4a0bfc6d-d753-4c24-b760-3afb506c824e\") "
Sep 04 23:53:08.976741 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.976712    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqdp8\" (UniqueName: \"kubernetes.io/projected/7183cdd7-779c-434a-98e1-d9952bdf6ee9-kube-api-access-tqdp8\") pod \"7183cdd7-779c-434a-98e1-d9952bdf6ee9\" (UID: \"7183cdd7-779c-434a-98e1-d9952bdf6ee9\") "
Sep 04 23:53:08.976993 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.976964    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a0bfc6d-d753-4c24-b760-3afb506c824e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4a0bfc6d-d753-4c24-b760-3afb506c824e" (UID: "4a0bfc6d-d753-4c24-b760-3afb506c824e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:53:08.977085 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.976996    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7183cdd7-779c-434a-98e1-d9952bdf6ee9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7183cdd7-779c-434a-98e1-d9952bdf6ee9" (UID: "7183cdd7-779c-434a-98e1-d9952bdf6ee9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:53:08.978946 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.978918    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7183cdd7-779c-434a-98e1-d9952bdf6ee9-kube-api-access-tqdp8" (OuterVolumeSpecName: "kube-api-access-tqdp8") pod "7183cdd7-779c-434a-98e1-d9952bdf6ee9" (UID: "7183cdd7-779c-434a-98e1-d9952bdf6ee9"). InnerVolumeSpecName "kube-api-access-tqdp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:53:08.979166 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:08.979142    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a0bfc6d-d753-4c24-b760-3afb506c824e-kube-api-access-c25x2" (OuterVolumeSpecName: "kube-api-access-c25x2") pod "4a0bfc6d-d753-4c24-b760-3afb506c824e" (UID: "4a0bfc6d-d753-4c24-b760-3afb506c824e"). InnerVolumeSpecName "kube-api-access-c25x2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:53:09.031360 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.031333    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:09.077599 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.077530    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3aa4556-4dd7-4ce6-95db-2a9511fb951e-maas-api-service-ca\") pod \"a3aa4556-4dd7-4ce6-95db-2a9511fb951e\" (UID: \"a3aa4556-4dd7-4ce6-95db-2a9511fb951e\") "
Sep 04 23:53:09.077754 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.077742    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c25x2\" (UniqueName: \"kubernetes.io/projected/4a0bfc6d-d753-4c24-b760-3afb506c824e-kube-api-access-c25x2\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:53:09.077794 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.077756    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqdp8\" (UniqueName: \"kubernetes.io/projected/7183cdd7-779c-434a-98e1-d9952bdf6ee9-kube-api-access-tqdp8\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:53:09.077794 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.077766    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a0bfc6d-d753-4c24-b760-3afb506c824e-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:53:09.077794 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.077775    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7183cdd7-779c-434a-98e1-d9952bdf6ee9-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:53:09.077888 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.077840    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3aa4556-4dd7-4ce6-95db-2a9511fb951e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a3aa4556-4dd7-4ce6-95db-2a9511fb951e" (UID: "a3aa4556-4dd7-4ce6-95db-2a9511fb951e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:53:09.179265 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.179226    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3aa4556-4dd7-4ce6-95db-2a9511fb951e-maas-api-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:53:09.812657 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.812569    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8"
Sep 04 23:53:09.812657 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.812576    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8" event={"ID":"a3aa4556-4dd7-4ce6-95db-2a9511fb951e","Type":"ContainerDied","Data":"ee0f45de67ed1d52781294aa844fc3f59ddfb74e78e7eeb36799cba7572fb77c"}
Sep 04 23:53:09.812657 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.812625    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee0f45de67ed1d52781294aa844fc3f59ddfb74e78e7eeb36799cba7572fb77c"
Sep 04 23:53:09.814078 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.814049    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"
Sep 04 23:53:09.814214 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.814049    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk" event={"ID":"7183cdd7-779c-434a-98e1-d9952bdf6ee9","Type":"ContainerDied","Data":"e1aca1106598f1f42581cf68f15ea637fb78bd1622adc5c8739a42cf4c9086f9"}
Sep 04 23:53:09.814214 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.814175    2564 scope.go:117] "RemoveContainer" containerID="c149315714a66abfc22bca92c98226f6ded4f8484e4efa0954210cd8c37bca47"
Sep 04 23:53:09.815643 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.815539    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n" event={"ID":"4a0bfc6d-d753-4c24-b760-3afb506c824e","Type":"ContainerDied","Data":"efcfae56723e3a41b54cfa1257138e3deabd43cb5d4217b4c8c10b2a3a78ee99"}
Sep 04 23:53:09.815643 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.815554    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"
Sep 04 23:53:09.823482 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.823466    2564 scope.go:117] "RemoveContainer" containerID="cf131e49909d32f53a40e37d09cfbd26bb6701d7b662042cafe5b3bd6a9c844a"
Sep 04 23:53:09.841439 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.841414    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"]
Sep 04 23:53:09.846552 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.846531    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-724e1f-6d648c7fc-52sgk"]
Sep 04 23:53:09.856175 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.856156    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"]
Sep 04 23:53:09.859528 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:09.859510    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-724e1f-67c89bd94-wb96n"]
Sep 04 23:53:10.176232 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:10.176204    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78" path="/var/lib/kubelet/pods/13bfc0cc-6bce-49e8-bcd5-7caff7bd8d78/volumes"
Sep 04 23:53:10.176553 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:10.176540    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a0bfc6d-d753-4c24-b760-3afb506c824e" path="/var/lib/kubelet/pods/4a0bfc6d-d753-4c24-b760-3afb506c824e/volumes"
Sep 04 23:53:10.176838 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:10.176827    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7183cdd7-779c-434a-98e1-d9952bdf6ee9" path="/var/lib/kubelet/pods/7183cdd7-779c-434a-98e1-d9952bdf6ee9/volumes"
Sep 04 23:53:14.701351 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:14.701268    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:14.754136 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:14.754109    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:15.321259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:15.321221    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:15.467691 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:15.467661    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:15.767405 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:15.767370    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:16.026041 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:16.025966    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:16.125532 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:16.125508    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:16.655155 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:16.655116    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:16.823053 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:16.823018    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8"]
Sep 04 23:53:16.825192 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:16.825169    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f4c99d38-ccwn8"]
Sep 04 23:53:17.021366 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:17.021294    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:17.081286 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:17.081249    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:17.169762 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:17.169739    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:18.174824 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:18.174792    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3aa4556-4dd7-4ce6-95db-2a9511fb951e" path="/var/lib/kubelet/pods/a3aa4556-4dd7-4ce6-95db-2a9511fb951e/volumes"
Sep 04 23:53:18.264870 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:18.264835    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:18.780561 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:18.780533    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:18.877088 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:18.877052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:19.910295 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:19.910247    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:20.012653 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.012623    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:20.118638 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.118610    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:20.385613 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.385573    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:20.479787 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.479753    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:20.519268 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.519237    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"]
Sep 04 23:53:20.519635 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.519621    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a0bfc6d-d753-4c24-b760-3afb506c824e" containerName="payload-pre-processing"
Sep 04 23:53:20.519679 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.519636    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a0bfc6d-d753-4c24-b760-3afb506c824e" containerName="payload-pre-processing"
Sep 04 23:53:20.519679 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.519655    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3aa4556-4dd7-4ce6-95db-2a9511fb951e" containerName="revoke-keys"
Sep 04 23:53:20.519679 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.519660    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3aa4556-4dd7-4ce6-95db-2a9511fb951e" containerName="revoke-keys"
Sep 04 23:53:20.519679 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.519672    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7183cdd7-779c-434a-98e1-d9952bdf6ee9" containerName="payload-processing"
Sep 04 23:53:20.519836 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.519681    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7183cdd7-779c-434a-98e1-d9952bdf6ee9" containerName="payload-processing"
Sep 04 23:53:20.519836 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.519737    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3aa4556-4dd7-4ce6-95db-2a9511fb951e" containerName="revoke-keys"
Sep 04 23:53:20.519836 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.519747    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7183cdd7-779c-434a-98e1-d9952bdf6ee9" containerName="payload-processing"
Sep 04 23:53:20.519836 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.519758    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a0bfc6d-d753-4c24-b760-3afb506c824e" containerName="payload-pre-processing"
Sep 04 23:53:20.522604 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.522589    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"
Sep 04 23:53:20.525233 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.525204    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-5245c1\""
Sep 04 23:53:20.525352 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.525211    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-5245c1-dockercfg-jr5sq\""
Sep 04 23:53:20.532160 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.532137    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"]
Sep 04 23:53:20.569142 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.569117    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c29a92-d22c-468a-aa6f-e312593f6ccd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs\" (UID: \"a2c29a92-d22c-468a-aa6f-e312593f6ccd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"
Sep 04 23:53:20.569287 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.569253    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n7zw\" (UniqueName: \"kubernetes.io/projected/a2c29a92-d22c-468a-aa6f-e312593f6ccd-kube-api-access-8n7zw\") pod \"payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs\" (UID: \"a2c29a92-d22c-468a-aa6f-e312593f6ccd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"
Sep 04 23:53:20.669928 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.669829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n7zw\" (UniqueName: \"kubernetes.io/projected/a2c29a92-d22c-468a-aa6f-e312593f6ccd-kube-api-access-8n7zw\") pod \"payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs\" (UID: \"a2c29a92-d22c-468a-aa6f-e312593f6ccd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"
Sep 04 23:53:20.669928 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.669918    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c29a92-d22c-468a-aa6f-e312593f6ccd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs\" (UID: \"a2c29a92-d22c-468a-aa6f-e312593f6ccd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"
Sep 04 23:53:20.670720 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.670699    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c29a92-d22c-468a-aa6f-e312593f6ccd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs\" (UID: \"a2c29a92-d22c-468a-aa6f-e312593f6ccd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"
Sep 04 23:53:20.677522 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.677503    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n7zw\" (UniqueName: \"kubernetes.io/projected/a2c29a92-d22c-468a-aa6f-e312593f6ccd-kube-api-access-8n7zw\") pod \"payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs\" (UID: \"a2c29a92-d22c-468a-aa6f-e312593f6ccd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"
Sep 04 23:53:20.833808 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:20.833783    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"
Sep 04 23:53:21.103251 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:21.103227    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"]
Sep 04 23:53:21.105036 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:53:21.105010    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c29a92_d22c_468a_aa6f_e312593f6ccd.slice/crio-4908466ce2ff8593aac7ed3af2b830255475e1575e2941045087ddf85b9c4756 WatchSource:0}: Error finding container 4908466ce2ff8593aac7ed3af2b830255475e1575e2941045087ddf85b9c4756: Status 404 returned error can't find the container with id 4908466ce2ff8593aac7ed3af2b830255475e1575e2941045087ddf85b9c4756
Sep 04 23:53:21.873999 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:21.873961    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs" event={"ID":"a2c29a92-d22c-468a-aa6f-e312593f6ccd","Type":"ContainerStarted","Data":"246fbade8a6a33f099e45b39cbe28af9e2447283851ee3e55250077ea1c43ac7"}
Sep 04 23:53:21.874190 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:21.874010    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs" event={"ID":"a2c29a92-d22c-468a-aa6f-e312593f6ccd","Type":"ContainerStarted","Data":"4908466ce2ff8593aac7ed3af2b830255475e1575e2941045087ddf85b9c4756"}
Sep 04 23:53:21.874190 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:21.874079    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"
Sep 04 23:53:21.891777 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:21.891738    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs" podStartSLOduration=1.891725511 podStartE2EDuration="1.891725511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:53:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:53:21.890822945 +0000 UTC m=+990.138823476" watchObservedRunningTime="2026-09-04 23:53:21.891725511 +0000 UTC m=+990.139726008"
Sep 04 23:53:22.685455 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:22.685421    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:22.743958 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:22.743929    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:22.921438 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:22.921405    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:23.284464 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:23.284434    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:23.374838 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:23.374804    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:23.474294 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:23.474272    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:28.667391 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:28.667352    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:29.016150 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:29.016067    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:29.114190 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:29.114155    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:29.214796 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:29.214749    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:29.348661 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:29.348602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:32.882408 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:32.882380    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"
Sep 04 23:53:35.031534 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:35.031497    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:35.131387 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:35.131356    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:35.949034 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:35.948992    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:36.336892 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:36.336794    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:36.437018 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:36.436970    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:39.263672 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:39.263639    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:39.356951 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:39.356890    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:40.451876 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:40.451840    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:40.549346 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:40.549300    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:40.646877 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:40.646847    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:40.925767 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:40.925732    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:41.091779 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.091728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:41.240042 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.239970    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"]
Sep 04 23:53:41.244594 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.244576    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"
Sep 04 23:53:41.247204 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.247184    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-e35897-dockercfg-q7fkr\""
Sep 04 23:53:41.247299 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.247206    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-e35897\""
Sep 04 23:53:41.250695 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.250670    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"]
Sep 04 23:53:41.328278 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.328250    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8gv5\" (UniqueName: \"kubernetes.io/projected/80c32a2d-405b-4e21-b12d-7e94189a1ba8-kube-api-access-p8gv5\") pod \"payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp\" (UID: \"80c32a2d-405b-4e21-b12d-7e94189a1ba8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"
Sep 04 23:53:41.328410 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.328311    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c32a2d-405b-4e21-b12d-7e94189a1ba8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp\" (UID: \"80c32a2d-405b-4e21-b12d-7e94189a1ba8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"
Sep 04 23:53:41.428817 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.428788    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8gv5\" (UniqueName: \"kubernetes.io/projected/80c32a2d-405b-4e21-b12d-7e94189a1ba8-kube-api-access-p8gv5\") pod \"payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp\" (UID: \"80c32a2d-405b-4e21-b12d-7e94189a1ba8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"
Sep 04 23:53:41.428817 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.428862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c32a2d-405b-4e21-b12d-7e94189a1ba8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp\" (UID: \"80c32a2d-405b-4e21-b12d-7e94189a1ba8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"
Sep 04 23:53:41.429514 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.429494    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c32a2d-405b-4e21-b12d-7e94189a1ba8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp\" (UID: \"80c32a2d-405b-4e21-b12d-7e94189a1ba8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"
Sep 04 23:53:41.437391 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.437369    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8gv5\" (UniqueName: \"kubernetes.io/projected/80c32a2d-405b-4e21-b12d-7e94189a1ba8-kube-api-access-p8gv5\") pod \"payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp\" (UID: \"80c32a2d-405b-4e21-b12d-7e94189a1ba8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"
Sep 04 23:53:41.556530 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.556461    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"
Sep 04 23:53:41.598080 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.598001    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:41.828256 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.828235    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"]
Sep 04 23:53:41.830293 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:53:41.830268    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80c32a2d_405b_4e21_b12d_7e94189a1ba8.slice/crio-8f47b1945c25b45f714d89d4a29490e050b6502511352480263de8f473d52d0c WatchSource:0}: Error finding container 8f47b1945c25b45f714d89d4a29490e050b6502511352480263de8f473d52d0c: Status 404 returned error can't find the container with id 8f47b1945c25b45f714d89d4a29490e050b6502511352480263de8f473d52d0c
Sep 04 23:53:41.972416 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.972377    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp" event={"ID":"80c32a2d-405b-4e21-b12d-7e94189a1ba8","Type":"ContainerStarted","Data":"68669c96457e7b77a00e03adb02a02585faa11c5ec1e8516571ad2e23d68752e"}
Sep 04 23:53:41.972582 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.972438    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp" event={"ID":"80c32a2d-405b-4e21-b12d-7e94189a1ba8","Type":"ContainerStarted","Data":"8f47b1945c25b45f714d89d4a29490e050b6502511352480263de8f473d52d0c"}
Sep 04 23:53:41.972644 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.972610    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"
Sep 04 23:53:41.988826 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:41.988778    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp" podStartSLOduration=0.988765365 podStartE2EDuration="988.765365ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:53:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:53:41.987016159 +0000 UTC m=+1010.235016702" watchObservedRunningTime="2026-09-04 23:53:41.988765365 +0000 UTC m=+1010.236765881"
Sep 04 23:53:42.226219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:42.226192    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:42.928614 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:42.928577    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:43.701434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:43.701399    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:44.349832 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:44.349799    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:44.450727 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:44.450692    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:47.297265 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:47.297224    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:47.396413 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:47.396373    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:48.446174 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:48.446138    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:48.505460 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:48.505424    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:49.060877 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.060847    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"]
Sep 04 23:53:49.063949 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.063926    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"
Sep 04 23:53:49.066458 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.066438    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-88828d\""
Sep 04 23:53:49.066693 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.066677    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-88828d-dockercfg-bw92f\""
Sep 04 23:53:49.072655 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.072635    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"]
Sep 04 23:53:49.193278 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.193250    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr\" (UID: \"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"
Sep 04 23:53:49.193416 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.193291    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsldm\" (UniqueName: \"kubernetes.io/projected/2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e-kube-api-access-gsldm\") pod \"payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr\" (UID: \"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"
Sep 04 23:53:49.293798 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.293766    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr\" (UID: \"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"
Sep 04 23:53:49.293961 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.293820    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsldm\" (UniqueName: \"kubernetes.io/projected/2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e-kube-api-access-gsldm\") pod \"payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr\" (UID: \"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"
Sep 04 23:53:49.294443 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.294426    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr\" (UID: \"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"
Sep 04 23:53:49.301263 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.301245    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsldm\" (UniqueName: \"kubernetes.io/projected/2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e-kube-api-access-gsldm\") pod \"payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr\" (UID: \"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"
Sep 04 23:53:49.375377 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.375353    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"
Sep 04 23:53:49.466448 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.466416    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:49.567666 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.567634    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:53:49.643247 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:49.643215    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"]
Sep 04 23:53:49.646157 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:53:49.646122    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f00d1f4_5327_4a88_bcd4_0c52e8b22e7e.slice/crio-08e416f90b015d8b6b55e541fd2fa4b1b5f4f77c4ab2475c3c12cafe5e83b8b9 WatchSource:0}: Error finding container 08e416f90b015d8b6b55e541fd2fa4b1b5f4f77c4ab2475c3c12cafe5e83b8b9: Status 404 returned error can't find the container with id 08e416f90b015d8b6b55e541fd2fa4b1b5f4f77c4ab2475c3c12cafe5e83b8b9
Sep 04 23:53:50.014808 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:50.014716    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr" event={"ID":"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e","Type":"ContainerStarted","Data":"9fe08ffe948e53df184d876a0f3ed66101104c72d38fe353d4cc4607f9102167"}
Sep 04 23:53:50.014808 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:50.014783    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr" event={"ID":"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e","Type":"ContainerStarted","Data":"08e416f90b015d8b6b55e541fd2fa4b1b5f4f77c4ab2475c3c12cafe5e83b8b9"}
Sep 04 23:53:50.015031 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:50.014839    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"
Sep 04 23:53:50.031502 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:50.031451    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr" podStartSLOduration=1.031434942 podStartE2EDuration="1.031434942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:53:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:53:50.029685054 +0000 UTC m=+1018.277685594" watchObservedRunningTime="2026-09-04 23:53:50.031434942 +0000 UTC m=+1018.279435459"
Sep 04 23:53:52.980972 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:53:52.980937    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"
Sep 04 23:54:01.022273 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:01.022243    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"
Sep 04 23:54:01.424969 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:01.424929    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:04.209528 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:04.209486    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:04.595559 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:04.595474    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:04.808649 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:04.808616    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:05.058049 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:05.058018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:07.447878 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:07.447836    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:07.545355 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:07.545329    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:08.718367 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:08.718330    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:08.770510 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:08.770480    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:09.173553 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.173522    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:09.274836 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.274792    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:09.282216 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.282190    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"]
Sep 04 23:54:09.288016 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.287998    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"
Sep 04 23:54:09.290653 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.290633    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-f7eb8e\""
Sep 04 23:54:09.290799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.290716    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-f7eb8e-dockercfg-tdmnl\""
Sep 04 23:54:09.295226 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.295204    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"]
Sep 04 23:54:09.319214 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.319195    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"]
Sep 04 23:54:09.322237 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.322223    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"
Sep 04 23:54:09.332487 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.332464    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"]
Sep 04 23:54:09.350475 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.350449    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:09.452160 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.452102    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fflzj\" (UniqueName: \"kubernetes.io/projected/33e5df03-ba9b-4fef-a44e-d574a98141e1-kube-api-access-fflzj\") pod \"payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z\" (UID: \"33e5df03-ba9b-4fef-a44e-d574a98141e1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"
Sep 04 23:54:09.452160 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.452154    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfrh5\" (UniqueName: \"kubernetes.io/projected/6bead5ba-e3c5-419e-b172-2d35682a9b48-kube-api-access-gfrh5\") pod \"payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf\" (UID: \"6bead5ba-e3c5-419e-b172-2d35682a9b48\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"
Sep 04 23:54:09.452290 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.452180    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33e5df03-ba9b-4fef-a44e-d574a98141e1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z\" (UID: \"33e5df03-ba9b-4fef-a44e-d574a98141e1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"
Sep 04 23:54:09.452290 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.452233    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bead5ba-e3c5-419e-b172-2d35682a9b48-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf\" (UID: \"6bead5ba-e3c5-419e-b172-2d35682a9b48\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"
Sep 04 23:54:09.553563 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.553541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33e5df03-ba9b-4fef-a44e-d574a98141e1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z\" (UID: \"33e5df03-ba9b-4fef-a44e-d574a98141e1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"
Sep 04 23:54:09.553687 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.553622    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bead5ba-e3c5-419e-b172-2d35682a9b48-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf\" (UID: \"6bead5ba-e3c5-419e-b172-2d35682a9b48\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"
Sep 04 23:54:09.553687 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.553666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fflzj\" (UniqueName: \"kubernetes.io/projected/33e5df03-ba9b-4fef-a44e-d574a98141e1-kube-api-access-fflzj\") pod \"payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z\" (UID: \"33e5df03-ba9b-4fef-a44e-d574a98141e1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"
Sep 04 23:54:09.553811 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.553725    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfrh5\" (UniqueName: \"kubernetes.io/projected/6bead5ba-e3c5-419e-b172-2d35682a9b48-kube-api-access-gfrh5\") pod \"payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf\" (UID: \"6bead5ba-e3c5-419e-b172-2d35682a9b48\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"
Sep 04 23:54:09.554228 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.554208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33e5df03-ba9b-4fef-a44e-d574a98141e1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z\" (UID: \"33e5df03-ba9b-4fef-a44e-d574a98141e1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"
Sep 04 23:54:09.554343 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.554322    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bead5ba-e3c5-419e-b172-2d35682a9b48-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf\" (UID: \"6bead5ba-e3c5-419e-b172-2d35682a9b48\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"
Sep 04 23:54:09.561539 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.561513    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fflzj\" (UniqueName: \"kubernetes.io/projected/33e5df03-ba9b-4fef-a44e-d574a98141e1-kube-api-access-fflzj\") pod \"payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z\" (UID: \"33e5df03-ba9b-4fef-a44e-d574a98141e1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"
Sep 04 23:54:09.561747 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.561727    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfrh5\" (UniqueName: \"kubernetes.io/projected/6bead5ba-e3c5-419e-b172-2d35682a9b48-kube-api-access-gfrh5\") pod \"payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf\" (UID: \"6bead5ba-e3c5-419e-b172-2d35682a9b48\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"
Sep 04 23:54:09.591190 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.591163    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:09.598830 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.598812    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"
Sep 04 23:54:09.631739 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.631715    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"
Sep 04 23:54:09.873084 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.873057    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"]
Sep 04 23:54:09.875240 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:54:09.875215    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bead5ba_e3c5_419e_b172_2d35682a9b48.slice/crio-b474f4fdbe63a5b70b1e1eac015cb6851d9dd215002e5c3cd17bee8b3e1c3b57 WatchSource:0}: Error finding container b474f4fdbe63a5b70b1e1eac015cb6851d9dd215002e5c3cd17bee8b3e1c3b57: Status 404 returned error can't find the container with id b474f4fdbe63a5b70b1e1eac015cb6851d9dd215002e5c3cd17bee8b3e1c3b57
Sep 04 23:54:09.917395 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:09.917375    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"]
Sep 04 23:54:09.918822 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:54:09.918792    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e5df03_ba9b_4fef_a44e_d574a98141e1.slice/crio-70d4b9f391ec3713adbbc0bade455ccb1af8cce5edd62ab9204410cf08327a43 WatchSource:0}: Error finding container 70d4b9f391ec3713adbbc0bade455ccb1af8cce5edd62ab9204410cf08327a43: Status 404 returned error can't find the container with id 70d4b9f391ec3713adbbc0bade455ccb1af8cce5edd62ab9204410cf08327a43
Sep 04 23:54:10.111739 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:10.111707    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf" event={"ID":"6bead5ba-e3c5-419e-b172-2d35682a9b48","Type":"ContainerStarted","Data":"b15de6052048c096f124737af856bd45f9caf9993a8b214881f95524c7c0a8ee"}
Sep 04 23:54:10.111958 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:10.111763    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf" event={"ID":"6bead5ba-e3c5-419e-b172-2d35682a9b48","Type":"ContainerStarted","Data":"b474f4fdbe63a5b70b1e1eac015cb6851d9dd215002e5c3cd17bee8b3e1c3b57"}
Sep 04 23:54:10.111958 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:10.111792    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"
Sep 04 23:54:10.113356 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:10.113325    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z" event={"ID":"33e5df03-ba9b-4fef-a44e-d574a98141e1","Type":"ContainerStarted","Data":"f544892ea5c1ebccacdabc2abb7582387974abe5dbf2982a09d4abe13c75cbc8"}
Sep 04 23:54:10.113488 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:10.113370    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z" event={"ID":"33e5df03-ba9b-4fef-a44e-d574a98141e1","Type":"ContainerStarted","Data":"70d4b9f391ec3713adbbc0bade455ccb1af8cce5edd62ab9204410cf08327a43"}
Sep 04 23:54:10.113488 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:10.113398    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"
Sep 04 23:54:10.126740 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:10.126649    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf" podStartSLOduration=1.126632446 podStartE2EDuration="1.126632446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:54:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:54:10.126547799 +0000 UTC m=+1038.374548342" watchObservedRunningTime="2026-09-04 23:54:10.126632446 +0000 UTC m=+1038.374632963"
Sep 04 23:54:10.143575 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:10.143535    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z" podStartSLOduration=1.143520893 podStartE2EDuration="1.143520893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:54:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:54:10.143093074 +0000 UTC m=+1038.391093615" watchObservedRunningTime="2026-09-04 23:54:10.143520893 +0000 UTC m=+1038.391521409"
Sep 04 23:54:10.390762 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:10.390684    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:11.115979 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:11.115943    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:12.227558 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:12.227529    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:17.376315 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:17.376281    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:21.120538 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:21.120503    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"
Sep 04 23:54:21.121028 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:21.120583    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"
Sep 04 23:54:26.144104 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:26.144064    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:29.687404 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:29.687367    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:32.392084 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:32.392050    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:33.109062 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:33.109022    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7630-a93c6c"]
Sep 04 23:54:33.112472 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:33.112451    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-a93c6c"
Sep 04 23:54:33.115178 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:33.115152    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5vfk6\""
Sep 04 23:54:33.117508 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:33.117478    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7630-a93c6c"]
Sep 04 23:54:33.145962 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:33.145940    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtrrj\" (UniqueName: \"kubernetes.io/projected/12e4424e-87eb-46e8-befc-ab6651201395-kube-api-access-xtrrj\") pod \"test-curl-7630-a93c6c\" (UID: \"12e4424e-87eb-46e8-befc-ab6651201395\") " pod="opendatahub/test-curl-7630-a93c6c"
Sep 04 23:54:33.247116 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:33.247095    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtrrj\" (UniqueName: \"kubernetes.io/projected/12e4424e-87eb-46e8-befc-ab6651201395-kube-api-access-xtrrj\") pod \"test-curl-7630-a93c6c\" (UID: \"12e4424e-87eb-46e8-befc-ab6651201395\") " pod="opendatahub/test-curl-7630-a93c6c"
Sep 04 23:54:33.254710 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:33.254693    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtrrj\" (UniqueName: \"kubernetes.io/projected/12e4424e-87eb-46e8-befc-ab6651201395-kube-api-access-xtrrj\") pod \"test-curl-7630-a93c6c\" (UID: \"12e4424e-87eb-46e8-befc-ab6651201395\") " pod="opendatahub/test-curl-7630-a93c6c"
Sep 04 23:54:33.424384 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:33.424331    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-a93c6c"
Sep 04 23:54:33.693236 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:33.693212    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7630-a93c6c"]
Sep 04 23:54:33.695005 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:54:33.694977    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12e4424e_87eb_46e8_befc_ab6651201395.slice/crio-b2c3cd8cf20dcc557d15eea2ab80f92d9814c53a20b697468f881777495ccd21 WatchSource:0}: Error finding container b2c3cd8cf20dcc557d15eea2ab80f92d9814c53a20b697468f881777495ccd21: Status 404 returned error can't find the container with id b2c3cd8cf20dcc557d15eea2ab80f92d9814c53a20b697468f881777495ccd21
Sep 04 23:54:34.231153 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:34.231120    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7630-a93c6c" event={"ID":"12e4424e-87eb-46e8-befc-ab6651201395","Type":"ContainerStarted","Data":"1fbe70011011285abc88918937591868d4204dea9430a2b45c4fa61d444083ef"}
Sep 04 23:54:34.231309 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:34.231163    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7630-a93c6c" event={"ID":"12e4424e-87eb-46e8-befc-ab6651201395","Type":"ContainerStarted","Data":"b2c3cd8cf20dcc557d15eea2ab80f92d9814c53a20b697468f881777495ccd21"}
Sep 04 23:54:34.244509 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:34.244465    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7630-a93c6c" podStartSLOduration=1.244453176 podStartE2EDuration="1.244453176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:54:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:54:34.243648959 +0000 UTC m=+1062.491649475" watchObservedRunningTime="2026-09-04 23:54:34.244453176 +0000 UTC m=+1062.492453691"
Sep 04 23:54:34.796553 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:34.796526    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7630-a93c6c"]
Sep 04 23:54:34.799126 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:34.799100    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7630-a93c6c"]
Sep 04 23:54:35.013194 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:35.013163    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7630-67fb5e"]
Sep 04 23:54:35.013623 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:35.013605    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12e4424e-87eb-46e8-befc-ab6651201395" containerName="test-curl-7630-a93c6c"
Sep 04 23:54:35.013715 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:35.013627    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="12e4424e-87eb-46e8-befc-ab6651201395" containerName="test-curl-7630-a93c6c"
Sep 04 23:54:35.013770 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:35.013743    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="12e4424e-87eb-46e8-befc-ab6651201395" containerName="test-curl-7630-a93c6c"
Sep 04 23:54:35.017029 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:35.017011    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-67fb5e"
Sep 04 23:54:35.022401 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:35.022377    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7630-67fb5e"]
Sep 04 23:54:35.063964 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:35.063943    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlhdm\" (UniqueName: \"kubernetes.io/projected/de55ccd3-88f7-43ae-86f7-a4e63b445ebe-kube-api-access-zlhdm\") pod \"test-curl-7630-67fb5e\" (UID: \"de55ccd3-88f7-43ae-86f7-a4e63b445ebe\") " pod="opendatahub/test-curl-7630-67fb5e"
Sep 04 23:54:35.164932 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:35.164887    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlhdm\" (UniqueName: \"kubernetes.io/projected/de55ccd3-88f7-43ae-86f7-a4e63b445ebe-kube-api-access-zlhdm\") pod \"test-curl-7630-67fb5e\" (UID: \"de55ccd3-88f7-43ae-86f7-a4e63b445ebe\") " pod="opendatahub/test-curl-7630-67fb5e"
Sep 04 23:54:35.173073 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:35.173054    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlhdm\" (UniqueName: \"kubernetes.io/projected/de55ccd3-88f7-43ae-86f7-a4e63b445ebe-kube-api-access-zlhdm\") pod \"test-curl-7630-67fb5e\" (UID: \"de55ccd3-88f7-43ae-86f7-a4e63b445ebe\") " pod="opendatahub/test-curl-7630-67fb5e"
Sep 04 23:54:35.236035 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:54:35.236006    2564 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7630-a93c6c" container="test-curl-7630-a93c6c"
Sep 04 23:54:35.327331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:35.327275    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-67fb5e"
Sep 04 23:54:35.598086 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:35.598062    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7630-67fb5e"]
Sep 04 23:54:35.599798 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:54:35.599771    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde55ccd3_88f7_43ae_86f7_a4e63b445ebe.slice/crio-7422a858222a38cd96850682f92f2fd778c6e5ad996696e5d196e7f01a311556 WatchSource:0}: Error finding container 7422a858222a38cd96850682f92f2fd778c6e5ad996696e5d196e7f01a311556: Status 404 returned error can't find the container with id 7422a858222a38cd96850682f92f2fd778c6e5ad996696e5d196e7f01a311556
Sep 04 23:54:36.241879 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:36.241846    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7630-67fb5e" event={"ID":"de55ccd3-88f7-43ae-86f7-a4e63b445ebe","Type":"ContainerStarted","Data":"11698cc4465efd86553259d62697c97363d635d0b3b992ec1b0ec8dd0964922d"}
Sep 04 23:54:36.242295 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:36.241886    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7630-67fb5e" event={"ID":"de55ccd3-88f7-43ae-86f7-a4e63b445ebe","Type":"ContainerStarted","Data":"7422a858222a38cd96850682f92f2fd778c6e5ad996696e5d196e7f01a311556"}
Sep 04 23:54:36.242295 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:36.241940    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7630-a93c6c" podUID="12e4424e-87eb-46e8-befc-ab6651201395" containerName="test-curl-7630-a93c6c" containerID="cri-o://1fbe70011011285abc88918937591868d4204dea9430a2b45c4fa61d444083ef" gracePeriod=2
Sep 04 23:54:36.255250 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:36.255207    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7630-67fb5e" podStartSLOduration=1.255193963 podStartE2EDuration="1.255193963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:54:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:54:36.254503698 +0000 UTC m=+1064.502504208" watchObservedRunningTime="2026-09-04 23:54:36.255193963 +0000 UTC m=+1064.503194519"
Sep 04 23:54:36.814176 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:36.814133    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7630-67fb5e"]
Sep 04 23:54:36.816596 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:36.816568    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7630-67fb5e"]
Sep 04 23:54:37.028332 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:37.028297    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7630-167a45"]
Sep 04 23:54:37.028799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:37.028780    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de55ccd3-88f7-43ae-86f7-a4e63b445ebe" containerName="test-curl-7630-67fb5e"
Sep 04 23:54:37.028892 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:37.028802    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="de55ccd3-88f7-43ae-86f7-a4e63b445ebe" containerName="test-curl-7630-67fb5e"
Sep 04 23:54:37.028986 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:37.028937    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="de55ccd3-88f7-43ae-86f7-a4e63b445ebe" containerName="test-curl-7630-67fb5e"
Sep 04 23:54:37.032012 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:37.031993    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-167a45"
Sep 04 23:54:37.035783 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:37.035764    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7630-167a45"]
Sep 04 23:54:37.080009 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:37.079975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gkfg\" (UniqueName: \"kubernetes.io/projected/337e8525-210a-47d8-98d1-bb71ecce7750-kube-api-access-7gkfg\") pod \"test-curl-7630-167a45\" (UID: \"337e8525-210a-47d8-98d1-bb71ecce7750\") " pod="opendatahub/test-curl-7630-167a45"
Sep 04 23:54:37.180603 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:37.180579    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gkfg\" (UniqueName: \"kubernetes.io/projected/337e8525-210a-47d8-98d1-bb71ecce7750-kube-api-access-7gkfg\") pod \"test-curl-7630-167a45\" (UID: \"337e8525-210a-47d8-98d1-bb71ecce7750\") " pod="opendatahub/test-curl-7630-167a45"
Sep 04 23:54:37.188459 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:37.188438    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gkfg\" (UniqueName: \"kubernetes.io/projected/337e8525-210a-47d8-98d1-bb71ecce7750-kube-api-access-7gkfg\") pod \"test-curl-7630-167a45\" (UID: \"337e8525-210a-47d8-98d1-bb71ecce7750\") " pod="opendatahub/test-curl-7630-167a45"
Sep 04 23:54:37.246964 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:54:37.246937    2564 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7630-67fb5e" container="test-curl-7630-67fb5e"
Sep 04 23:54:37.341937 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:37.341878    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-167a45"
Sep 04 23:54:37.615109 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:37.615085    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7630-167a45"]
Sep 04 23:54:37.616659 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:54:37.616633    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod337e8525_210a_47d8_98d1_bb71ecce7750.slice/crio-9171f75b2dfceaa731c073efdae99836f4373144e0f9f5ac230d0ada1582dab4 WatchSource:0}: Error finding container 9171f75b2dfceaa731c073efdae99836f4373144e0f9f5ac230d0ada1582dab4: Status 404 returned error can't find the container with id 9171f75b2dfceaa731c073efdae99836f4373144e0f9f5ac230d0ada1582dab4
Sep 04 23:54:38.253795 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:38.253768    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7630-167a45" event={"ID":"337e8525-210a-47d8-98d1-bb71ecce7750","Type":"ContainerStarted","Data":"c23b4a90a1152fd8cafba7bb968166a6479833d3fe8092786e79a704dc91e69d"}
Sep 04 23:54:38.254174 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:38.253812    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7630-167a45" event={"ID":"337e8525-210a-47d8-98d1-bb71ecce7750","Type":"ContainerStarted","Data":"9171f75b2dfceaa731c073efdae99836f4373144e0f9f5ac230d0ada1582dab4"}
Sep 04 23:54:38.254174 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:38.253844    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7630-67fb5e" podUID="de55ccd3-88f7-43ae-86f7-a4e63b445ebe" containerName="test-curl-7630-67fb5e" containerID="cri-o://11698cc4465efd86553259d62697c97363d635d0b3b992ec1b0ec8dd0964922d" gracePeriod=2
Sep 04 23:54:38.267262 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:38.267221    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7630-167a45" podStartSLOduration=1.267207054 podStartE2EDuration="1.267207054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:54:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:54:38.266737973 +0000 UTC m=+1066.514738488" watchObservedRunningTime="2026-09-04 23:54:38.267207054 +0000 UTC m=+1066.515207570"
Sep 04 23:54:38.424690 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:38.424669    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-a93c6c"
Sep 04 23:54:38.490360 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:38.490336    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtrrj\" (UniqueName: \"kubernetes.io/projected/12e4424e-87eb-46e8-befc-ab6651201395-kube-api-access-xtrrj\") pod \"12e4424e-87eb-46e8-befc-ab6651201395\" (UID: \"12e4424e-87eb-46e8-befc-ab6651201395\") "
Sep 04 23:54:38.492307 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:38.492271    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12e4424e-87eb-46e8-befc-ab6651201395-kube-api-access-xtrrj" (OuterVolumeSpecName: "kube-api-access-xtrrj") pod "12e4424e-87eb-46e8-befc-ab6651201395" (UID: "12e4424e-87eb-46e8-befc-ab6651201395"). InnerVolumeSpecName "kube-api-access-xtrrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:54:38.591065 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:38.591043    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtrrj\" (UniqueName: \"kubernetes.io/projected/12e4424e-87eb-46e8-befc-ab6651201395-kube-api-access-xtrrj\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:54:38.827155 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:38.827093    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7630-167a45"]
Sep 04 23:54:38.830512 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:38.830479    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7630-167a45"]
Sep 04 23:54:39.045917 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.045880    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7630-b3151d"]
Sep 04 23:54:39.046287 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.046274    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="337e8525-210a-47d8-98d1-bb71ecce7750" containerName="test-curl-7630-167a45"
Sep 04 23:54:39.046346 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.046290    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="337e8525-210a-47d8-98d1-bb71ecce7750" containerName="test-curl-7630-167a45"
Sep 04 23:54:39.046383 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.046367    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="337e8525-210a-47d8-98d1-bb71ecce7750" containerName="test-curl-7630-167a45"
Sep 04 23:54:39.049520 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.049503    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-b3151d"
Sep 04 23:54:39.053881 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.053861    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7630-b3151d"]
Sep 04 23:54:39.094769 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.094713    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mxpt\" (UniqueName: \"kubernetes.io/projected/760e2a1a-1a5a-4e1f-8707-4d946eab1d3f-kube-api-access-7mxpt\") pod \"test-curl-7630-b3151d\" (UID: \"760e2a1a-1a5a-4e1f-8707-4d946eab1d3f\") " pod="opendatahub/test-curl-7630-b3151d"
Sep 04 23:54:39.195196 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.195175    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mxpt\" (UniqueName: \"kubernetes.io/projected/760e2a1a-1a5a-4e1f-8707-4d946eab1d3f-kube-api-access-7mxpt\") pod \"test-curl-7630-b3151d\" (UID: \"760e2a1a-1a5a-4e1f-8707-4d946eab1d3f\") " pod="opendatahub/test-curl-7630-b3151d"
Sep 04 23:54:39.205093 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.205068    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mxpt\" (UniqueName: \"kubernetes.io/projected/760e2a1a-1a5a-4e1f-8707-4d946eab1d3f-kube-api-access-7mxpt\") pod \"test-curl-7630-b3151d\" (UID: \"760e2a1a-1a5a-4e1f-8707-4d946eab1d3f\") " pod="opendatahub/test-curl-7630-b3151d"
Sep 04 23:54:39.260004 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.259978    2564 generic.go:356] "Generic (PLEG): container finished" podID="12e4424e-87eb-46e8-befc-ab6651201395" containerID="1fbe70011011285abc88918937591868d4204dea9430a2b45c4fa61d444083ef" exitCode=137
Sep 04 23:54:39.260323 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.260018    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-a93c6c"
Sep 04 23:54:39.260323 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.260072    2564 scope.go:117] "RemoveContainer" containerID="1fbe70011011285abc88918937591868d4204dea9430a2b45c4fa61d444083ef"
Sep 04 23:54:39.260323 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:54:39.260316    2564 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7630-167a45" container="test-curl-7630-167a45"
Sep 04 23:54:39.269473 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.269450    2564 scope.go:117] "RemoveContainer" containerID="1fbe70011011285abc88918937591868d4204dea9430a2b45c4fa61d444083ef"
Sep 04 23:54:39.269791 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:54:39.269772    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fbe70011011285abc88918937591868d4204dea9430a2b45c4fa61d444083ef\": container with ID starting with 1fbe70011011285abc88918937591868d4204dea9430a2b45c4fa61d444083ef not found: ID does not exist" containerID="1fbe70011011285abc88918937591868d4204dea9430a2b45c4fa61d444083ef"
Sep 04 23:54:39.269860 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.269803    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fbe70011011285abc88918937591868d4204dea9430a2b45c4fa61d444083ef"} err="failed to get container status \"1fbe70011011285abc88918937591868d4204dea9430a2b45c4fa61d444083ef\": rpc error: code = NotFound desc = could not find container \"1fbe70011011285abc88918937591868d4204dea9430a2b45c4fa61d444083ef\": container with ID starting with 1fbe70011011285abc88918937591868d4204dea9430a2b45c4fa61d444083ef not found: ID does not exist"
Sep 04 23:54:39.271309 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:54:39.271288    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7630-a93c6c\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="12e4424e-87eb-46e8-befc-ab6651201395" pod="opendatahub/test-curl-7630-a93c6c"
Sep 04 23:54:39.360148 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.360129    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-b3151d"
Sep 04 23:54:39.626436 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:39.626416    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7630-b3151d"]
Sep 04 23:54:39.628485 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:54:39.628464    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod760e2a1a_1a5a_4e1f_8707_4d946eab1d3f.slice/crio-7d9606df0e506dc24c23091ce562f1a1c4307695047785a85d0864cfcd4b9335 WatchSource:0}: Error finding container 7d9606df0e506dc24c23091ce562f1a1c4307695047785a85d0864cfcd4b9335: Status 404 returned error can't find the container with id 7d9606df0e506dc24c23091ce562f1a1c4307695047785a85d0864cfcd4b9335
Sep 04 23:54:40.176017 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:40.175985    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12e4424e-87eb-46e8-befc-ab6651201395" path="/var/lib/kubelet/pods/12e4424e-87eb-46e8-befc-ab6651201395/volumes"
Sep 04 23:54:40.269684 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:40.269657    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7630-b3151d" event={"ID":"760e2a1a-1a5a-4e1f-8707-4d946eab1d3f","Type":"ContainerStarted","Data":"89410183de834f6ec94c9f85190c5465f1b4f1fdb6adb8238dd4b6d337625624"}
Sep 04 23:54:40.270093 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:40.269698    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7630-b3151d" event={"ID":"760e2a1a-1a5a-4e1f-8707-4d946eab1d3f","Type":"ContainerStarted","Data":"7d9606df0e506dc24c23091ce562f1a1c4307695047785a85d0864cfcd4b9335"}
Sep 04 23:54:40.270093 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:40.269786    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7630-167a45" podUID="337e8525-210a-47d8-98d1-bb71ecce7750" containerName="test-curl-7630-167a45" containerID="cri-o://c23b4a90a1152fd8cafba7bb968166a6479833d3fe8092786e79a704dc91e69d" gracePeriod=2
Sep 04 23:54:40.285207 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:40.285161    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7630-b3151d" podStartSLOduration=1.2851470520000001 podStartE2EDuration="1.285147052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:54:40.282972237 +0000 UTC m=+1068.530972973" watchObservedRunningTime="2026-09-04 23:54:40.285147052 +0000 UTC m=+1068.533147568"
Sep 04 23:54:40.403858 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:40.403835    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-67fb5e"
Sep 04 23:54:40.506868 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:40.506816    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlhdm\" (UniqueName: \"kubernetes.io/projected/de55ccd3-88f7-43ae-86f7-a4e63b445ebe-kube-api-access-zlhdm\") pod \"de55ccd3-88f7-43ae-86f7-a4e63b445ebe\" (UID: \"de55ccd3-88f7-43ae-86f7-a4e63b445ebe\") "
Sep 04 23:54:40.508810 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:40.508789    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de55ccd3-88f7-43ae-86f7-a4e63b445ebe-kube-api-access-zlhdm" (OuterVolumeSpecName: "kube-api-access-zlhdm") pod "de55ccd3-88f7-43ae-86f7-a4e63b445ebe" (UID: "de55ccd3-88f7-43ae-86f7-a4e63b445ebe"). InnerVolumeSpecName "kube-api-access-zlhdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:54:40.607872 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:40.607850    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlhdm\" (UniqueName: \"kubernetes.io/projected/de55ccd3-88f7-43ae-86f7-a4e63b445ebe-kube-api-access-zlhdm\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:54:40.837478 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:40.837410    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7630-b3151d"]
Sep 04 23:54:40.839797 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:40.839767    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7630-b3151d"]
Sep 04 23:54:41.275884 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:41.275850    2564 generic.go:356] "Generic (PLEG): container finished" podID="de55ccd3-88f7-43ae-86f7-a4e63b445ebe" containerID="11698cc4465efd86553259d62697c97363d635d0b3b992ec1b0ec8dd0964922d" exitCode=137
Sep 04 23:54:41.276331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:41.275920    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-67fb5e"
Sep 04 23:54:41.276331 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:41.275953    2564 scope.go:117] "RemoveContainer" containerID="11698cc4465efd86553259d62697c97363d635d0b3b992ec1b0ec8dd0964922d"
Sep 04 23:54:41.276331 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:54:41.276279    2564 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7630-b3151d" container="test-curl-7630-b3151d"
Sep 04 23:54:41.284980 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:41.284959    2564 scope.go:117] "RemoveContainer" containerID="11698cc4465efd86553259d62697c97363d635d0b3b992ec1b0ec8dd0964922d"
Sep 04 23:54:41.285264 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:54:41.285245    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11698cc4465efd86553259d62697c97363d635d0b3b992ec1b0ec8dd0964922d\": container with ID starting with 11698cc4465efd86553259d62697c97363d635d0b3b992ec1b0ec8dd0964922d not found: ID does not exist" containerID="11698cc4465efd86553259d62697c97363d635d0b3b992ec1b0ec8dd0964922d"
Sep 04 23:54:41.285315 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:41.285271    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11698cc4465efd86553259d62697c97363d635d0b3b992ec1b0ec8dd0964922d"} err="failed to get container status \"11698cc4465efd86553259d62697c97363d635d0b3b992ec1b0ec8dd0964922d\": rpc error: code = NotFound desc = could not find container \"11698cc4465efd86553259d62697c97363d635d0b3b992ec1b0ec8dd0964922d\": container with ID starting with 11698cc4465efd86553259d62697c97363d635d0b3b992ec1b0ec8dd0964922d not found: ID does not exist"
Sep 04 23:54:41.286173 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:54:41.286152    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7630-67fb5e\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="de55ccd3-88f7-43ae-86f7-a4e63b445ebe" pod="opendatahub/test-curl-7630-67fb5e"
Sep 04 23:54:41.401952 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:41.401926    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:41.685501 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:41.685448    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:42.172695 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:54:42.172654    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7630-67fb5e\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="de55ccd3-88f7-43ae-86f7-a4e63b445ebe" pod="opendatahub/test-curl-7630-67fb5e"
Sep 04 23:54:42.174763 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:42.174740    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de55ccd3-88f7-43ae-86f7-a4e63b445ebe" path="/var/lib/kubelet/pods/de55ccd3-88f7-43ae-86f7-a4e63b445ebe/volumes"
Sep 04 23:54:42.191647 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:42.191612    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:42.282344 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:42.282310    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7630-b3151d" podUID="760e2a1a-1a5a-4e1f-8707-4d946eab1d3f" containerName="test-curl-7630-b3151d" containerID="cri-o://89410183de834f6ec94c9f85190c5465f1b4f1fdb6adb8238dd4b6d337625624" gracePeriod=2
Sep 04 23:54:42.461637 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:42.461608    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-167a45"
Sep 04 23:54:42.522841 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:42.522819    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gkfg\" (UniqueName: \"kubernetes.io/projected/337e8525-210a-47d8-98d1-bb71ecce7750-kube-api-access-7gkfg\") pod \"337e8525-210a-47d8-98d1-bb71ecce7750\" (UID: \"337e8525-210a-47d8-98d1-bb71ecce7750\") "
Sep 04 23:54:42.524915 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:42.524876    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/337e8525-210a-47d8-98d1-bb71ecce7750-kube-api-access-7gkfg" (OuterVolumeSpecName: "kube-api-access-7gkfg") pod "337e8525-210a-47d8-98d1-bb71ecce7750" (UID: "337e8525-210a-47d8-98d1-bb71ecce7750"). InnerVolumeSpecName "kube-api-access-7gkfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:54:42.623888 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:42.623869    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gkfg\" (UniqueName: \"kubernetes.io/projected/337e8525-210a-47d8-98d1-bb71ecce7750-kube-api-access-7gkfg\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:54:43.287944 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:43.287890    2564 generic.go:356] "Generic (PLEG): container finished" podID="337e8525-210a-47d8-98d1-bb71ecce7750" containerID="c23b4a90a1152fd8cafba7bb968166a6479833d3fe8092786e79a704dc91e69d" exitCode=137
Sep 04 23:54:43.288352 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:43.287958    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-167a45"
Sep 04 23:54:43.288352 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:43.288006    2564 scope.go:117] "RemoveContainer" containerID="c23b4a90a1152fd8cafba7bb968166a6479833d3fe8092786e79a704dc91e69d"
Sep 04 23:54:43.296982 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:43.296966    2564 scope.go:117] "RemoveContainer" containerID="c23b4a90a1152fd8cafba7bb968166a6479833d3fe8092786e79a704dc91e69d"
Sep 04 23:54:43.297261 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:54:43.297240    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c23b4a90a1152fd8cafba7bb968166a6479833d3fe8092786e79a704dc91e69d\": container with ID starting with c23b4a90a1152fd8cafba7bb968166a6479833d3fe8092786e79a704dc91e69d not found: ID does not exist" containerID="c23b4a90a1152fd8cafba7bb968166a6479833d3fe8092786e79a704dc91e69d"
Sep 04 23:54:43.297332 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:43.297267    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23b4a90a1152fd8cafba7bb968166a6479833d3fe8092786e79a704dc91e69d"} err="failed to get container status \"c23b4a90a1152fd8cafba7bb968166a6479833d3fe8092786e79a704dc91e69d\": rpc error: code = NotFound desc = could not find container \"c23b4a90a1152fd8cafba7bb968166a6479833d3fe8092786e79a704dc91e69d\": container with ID starting with c23b4a90a1152fd8cafba7bb968166a6479833d3fe8092786e79a704dc91e69d not found: ID does not exist"
Sep 04 23:54:43.298071 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:54:43.298050    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7630-167a45\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="337e8525-210a-47d8-98d1-bb71ecce7750" pod="opendatahub/test-curl-7630-167a45"
Sep 04 23:54:44.176085 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:44.176051    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="337e8525-210a-47d8-98d1-bb71ecce7750" path="/var/lib/kubelet/pods/337e8525-210a-47d8-98d1-bb71ecce7750/volumes"
Sep 04 23:54:44.480779 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:44.480758    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-b3151d"
Sep 04 23:54:44.540280 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:44.540250    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mxpt\" (UniqueName: \"kubernetes.io/projected/760e2a1a-1a5a-4e1f-8707-4d946eab1d3f-kube-api-access-7mxpt\") pod \"760e2a1a-1a5a-4e1f-8707-4d946eab1d3f\" (UID: \"760e2a1a-1a5a-4e1f-8707-4d946eab1d3f\") "
Sep 04 23:54:44.542306 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:44.542287    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/760e2a1a-1a5a-4e1f-8707-4d946eab1d3f-kube-api-access-7mxpt" (OuterVolumeSpecName: "kube-api-access-7mxpt") pod "760e2a1a-1a5a-4e1f-8707-4d946eab1d3f" (UID: "760e2a1a-1a5a-4e1f-8707-4d946eab1d3f"). InnerVolumeSpecName "kube-api-access-7mxpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:54:44.640867 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:44.640844    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mxpt\" (UniqueName: \"kubernetes.io/projected/760e2a1a-1a5a-4e1f-8707-4d946eab1d3f-kube-api-access-7mxpt\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:54:44.686303 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:44.686272    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:44.784930 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:44.784832    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:45.300187 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:45.300157    2564 generic.go:356] "Generic (PLEG): container finished" podID="760e2a1a-1a5a-4e1f-8707-4d946eab1d3f" containerID="89410183de834f6ec94c9f85190c5465f1b4f1fdb6adb8238dd4b6d337625624" exitCode=137
Sep 04 23:54:45.300394 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:45.300194    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7630-b3151d"
Sep 04 23:54:45.300394 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:45.300244    2564 scope.go:117] "RemoveContainer" containerID="89410183de834f6ec94c9f85190c5465f1b4f1fdb6adb8238dd4b6d337625624"
Sep 04 23:54:45.308633 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:45.308616    2564 scope.go:117] "RemoveContainer" containerID="89410183de834f6ec94c9f85190c5465f1b4f1fdb6adb8238dd4b6d337625624"
Sep 04 23:54:45.308911 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:54:45.308879    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89410183de834f6ec94c9f85190c5465f1b4f1fdb6adb8238dd4b6d337625624\": container with ID starting with 89410183de834f6ec94c9f85190c5465f1b4f1fdb6adb8238dd4b6d337625624 not found: ID does not exist" containerID="89410183de834f6ec94c9f85190c5465f1b4f1fdb6adb8238dd4b6d337625624"
Sep 04 23:54:45.308975 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:45.308924    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89410183de834f6ec94c9f85190c5465f1b4f1fdb6adb8238dd4b6d337625624"} err="failed to get container status \"89410183de834f6ec94c9f85190c5465f1b4f1fdb6adb8238dd4b6d337625624\": rpc error: code = NotFound desc = could not find container \"89410183de834f6ec94c9f85190c5465f1b4f1fdb6adb8238dd4b6d337625624\": container with ID starting with 89410183de834f6ec94c9f85190c5465f1b4f1fdb6adb8238dd4b6d337625624 not found: ID does not exist"
Sep 04 23:54:45.309728 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:54:45.309706    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7630-b3151d\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="760e2a1a-1a5a-4e1f-8707-4d946eab1d3f" pod="opendatahub/test-curl-7630-b3151d"
Sep 04 23:54:45.943651 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:45.943616    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:45.994188 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:45.994156    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:46.100395 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.100362    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:46.174061 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.174032    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="760e2a1a-1a5a-4e1f-8707-4d946eab1d3f" path="/var/lib/kubelet/pods/760e2a1a-1a5a-4e1f-8707-4d946eab1d3f/volumes"
Sep 04 23:54:46.302738 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.302666    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:46.407919 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.407864    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:54:46.415206 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.415179    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"]
Sep 04 23:54:46.415568 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.415555    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="760e2a1a-1a5a-4e1f-8707-4d946eab1d3f" containerName="test-curl-7630-b3151d"
Sep 04 23:54:46.415629 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.415570    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="760e2a1a-1a5a-4e1f-8707-4d946eab1d3f" containerName="test-curl-7630-b3151d"
Sep 04 23:54:46.415667 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.415642    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="760e2a1a-1a5a-4e1f-8707-4d946eab1d3f" containerName="test-curl-7630-b3151d"
Sep 04 23:54:46.419973 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.419958    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"
Sep 04 23:54:46.422681 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.422652    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-ede0daf1-dockercfg-ngsld\""
Sep 04 23:54:46.422798 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.422701    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-ede0daf1\""
Sep 04 23:54:46.427525 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.427503    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"]
Sep 04 23:54:46.558080 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.558006    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4qg9\" (UniqueName: \"kubernetes.io/projected/9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec-kube-api-access-b4qg9\") pod \"payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj\" (UID: \"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"
Sep 04 23:54:46.558217 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.558089    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj\" (UID: \"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"
Sep 04 23:54:46.659220 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.659188    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4qg9\" (UniqueName: \"kubernetes.io/projected/9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec-kube-api-access-b4qg9\") pod \"payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj\" (UID: \"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"
Sep 04 23:54:46.659378 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.659250    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj\" (UID: \"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"
Sep 04 23:54:46.659827 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.659808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj\" (UID: \"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"
Sep 04 23:54:46.666915 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.666869    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4qg9\" (UniqueName: \"kubernetes.io/projected/9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec-kube-api-access-b4qg9\") pod \"payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj\" (UID: \"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"
Sep 04 23:54:46.731942 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:46.731912    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"
Sep 04 23:54:47.002805 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:47.002778    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"]
Sep 04 23:54:47.004011 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:54:47.003987    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d6b6e8d_c6d8_408a_9020_37ea8a97a6ec.slice/crio-93f26ea70e2daf51b2267e57e30d0166ec53c58aeaf204dbf0269b13fa51b5d1 WatchSource:0}: Error finding container 93f26ea70e2daf51b2267e57e30d0166ec53c58aeaf204dbf0269b13fa51b5d1: Status 404 returned error can't find the container with id 93f26ea70e2daf51b2267e57e30d0166ec53c58aeaf204dbf0269b13fa51b5d1
Sep 04 23:54:47.313241 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:47.313160    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj" event={"ID":"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec","Type":"ContainerStarted","Data":"b77697b9c51be87d386ef4a19e943c2007ecde702a4e7a8117834e40584e45af"}
Sep 04 23:54:47.313241 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:47.313220    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj" event={"ID":"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec","Type":"ContainerStarted","Data":"93f26ea70e2daf51b2267e57e30d0166ec53c58aeaf204dbf0269b13fa51b5d1"}
Sep 04 23:54:47.313477 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:47.313282    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"
Sep 04 23:54:47.329548 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:47.329499    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj" podStartSLOduration=1.32948637 podStartE2EDuration="1.32948637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:54:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:54:47.327672919 +0000 UTC m=+1075.575673465" watchObservedRunningTime="2026-09-04 23:54:47.32948637 +0000 UTC m=+1075.577486885"
Sep 04 23:54:58.320175 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:54:58.320143    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"
Sep 04 23:55:06.516938 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.516878    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"]
Sep 04 23:55:06.521832 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.521812    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.524341 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.524317    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-88828d\"/\"openshift-service-ca.crt\""
Sep 04 23:55:06.525552 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.525532    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-88828d\"/\"default-dockercfg-w5fgz\""
Sep 04 23:55:06.525666 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.525618    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-88828d\"/\"kube-root-ca.crt\""
Sep 04 23:55:06.525666 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.525656    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-88828d\"/\"auth-test-model-88828d-kserve-self-signed-certs\""
Sep 04 23:55:06.528256 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.528238    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"]
Sep 04 23:55:06.545620 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.545593    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:55:06.614758 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.614735    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-dshm\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.614846 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.614775    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qgn6\" (UniqueName: \"kubernetes.io/projected/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-kube-api-access-6qgn6\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.614888 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.614869    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-home\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.614952 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.614927    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-tls-certs\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.614990 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.614956    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-model-cache\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.615032 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.615018    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-tmp-dir\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.716227 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.716200    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-tmp-dir\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.716335 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.716239    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-dshm\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.716335 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.716285    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qgn6\" (UniqueName: \"kubernetes.io/projected/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-kube-api-access-6qgn6\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.716458 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.716374    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-home\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.716458 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.716407    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-tls-certs\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.716458 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.716435    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-model-cache\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.716630 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.716607    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-tmp-dir\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.716736 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.716715    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-home\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.716799 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.716773    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-model-cache\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.718779 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.718761    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-dshm\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.718869 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.718854    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-tls-certs\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.723805 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.723788    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qgn6\" (UniqueName: \"kubernetes.io/projected/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-kube-api-access-6qgn6\") pod \"auth-test-model-88828d-kserve-76cfd49bb7-nmjk4\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") " pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:06.833139 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:06.833084    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:07.104719 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:07.104698    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"]
Sep 04 23:55:07.106862 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:55:07.106838    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd23936cb_9c57_44d9_ab96_f3a7e3a98fc3.slice/crio-62db3af87c6921952641d558a947029cd52458f94257e1e3fbf3c5374ce2080d WatchSource:0}: Error finding container 62db3af87c6921952641d558a947029cd52458f94257e1e3fbf3c5374ce2080d: Status 404 returned error can't find the container with id 62db3af87c6921952641d558a947029cd52458f94257e1e3fbf3c5374ce2080d
Sep 04 23:55:07.413265 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:07.413230    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4" event={"ID":"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3","Type":"ContainerStarted","Data":"c1a8ba7e7f615c62b4587e0c44878d9e21205e65f8350c975410ae1c355e6995"}
Sep 04 23:55:07.413422 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:07.413278    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4" event={"ID":"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3","Type":"ContainerStarted","Data":"62db3af87c6921952641d558a947029cd52458f94257e1e3fbf3c5374ce2080d"}
Sep 04 23:55:07.432053 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:07.432001    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4" podStartSLOduration=1.431987387 podStartE2EDuration="1.431987387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:55:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:55:07.430726609 +0000 UTC m=+1095.678727124" watchObservedRunningTime="2026-09-04 23:55:07.431987387 +0000 UTC m=+1095.679987903"
Sep 04 23:55:09.627030 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:09.626992    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:55:09.873083 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:09.873052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:55:16.833404 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:16.833353    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:16.833926 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:16.833443    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:16.846135 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:16.846112    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:17.473316 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:17.473284    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:55:24.512775 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.512733    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"]
Sep 04 23:55:24.516528 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.516502    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.520616 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.520594    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ede0daf1\"/\"default-dockercfg-p87zf\""
Sep 04 23:55:24.520946 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.520604    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ede0daf1\"/\"e2e-lifecycle-model-ede0daf1-kserve-self-signed-certs\""
Sep 04 23:55:24.521119 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.520604    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ede0daf1\"/\"openshift-service-ca.crt\""
Sep 04 23:55:24.521243 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.520725    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ede0daf1\"/\"kube-root-ca.crt\""
Sep 04 23:55:24.526666 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.526639    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"]
Sep 04 23:55:24.543375 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.543349    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:55:24.572705 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.572680    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-home\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.572824 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.572740    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-dshm\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.572824 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.572754    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-model-cache\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.572992 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.572825    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-tmp-dir\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.572992 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.572924    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/225f09a3-37de-478a-a216-1d6560948fad-tls-certs\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.572992 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.572979    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd2lq\" (UniqueName: \"kubernetes.io/projected/225f09a3-37de-478a-a216-1d6560948fad-kube-api-access-bd2lq\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.674057 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.674028    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/225f09a3-37de-478a-a216-1d6560948fad-tls-certs\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.674197 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.674070    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd2lq\" (UniqueName: \"kubernetes.io/projected/225f09a3-37de-478a-a216-1d6560948fad-kube-api-access-bd2lq\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.674197 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.674090    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-home\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.674197 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.674159    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-dshm\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.674197 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.674173    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-model-cache\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.674434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.674220    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-tmp-dir\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.674604 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.674583    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-tmp-dir\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.674604 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.674598    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-home\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.674716 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.674695    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-model-cache\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.676538 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.676513    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-dshm\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.676675 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.676656    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/225f09a3-37de-478a-a216-1d6560948fad-tls-certs\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.682281 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.682260    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd2lq\" (UniqueName: \"kubernetes.io/projected/225f09a3-37de-478a-a216-1d6560948fad-kube-api-access-bd2lq\") pod \"e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") " pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:24.828661 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:24.828576    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:25.100887 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:25.100864    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"]
Sep 04 23:55:25.103000 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:55:25.102973    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod225f09a3_37de_478a_a216_1d6560948fad.slice/crio-004b993accef1f00110b1ef6b76d472cc845ccb8baccd215535054cbf33d0994 WatchSource:0}: Error finding container 004b993accef1f00110b1ef6b76d472cc845ccb8baccd215535054cbf33d0994: Status 404 returned error can't find the container with id 004b993accef1f00110b1ef6b76d472cc845ccb8baccd215535054cbf33d0994
Sep 04 23:55:25.504057 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:25.504021    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz" event={"ID":"225f09a3-37de-478a-a216-1d6560948fad","Type":"ContainerStarted","Data":"03a20079a7f3e1da69fa8a0cd1a045357c7539a0f59b41d304911c2bb81df720"}
Sep 04 23:55:25.504057 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:25.504060    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz" event={"ID":"225f09a3-37de-478a-a216-1d6560948fad","Type":"ContainerStarted","Data":"004b993accef1f00110b1ef6b76d472cc845ccb8baccd215535054cbf33d0994"}
Sep 04 23:55:25.520333 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:25.520286    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz" podStartSLOduration=1.520272608 podStartE2EDuration="1.520272608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:55:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:55:25.518922445 +0000 UTC m=+1113.766922957" watchObservedRunningTime="2026-09-04 23:55:25.520272608 +0000 UTC m=+1113.768273124"
Sep 04 23:55:34.829549 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:34.829514    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:34.829549 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:34.829558    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:34.842040 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:34.842017    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:35.572970 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:35.572939    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:55:57.443625 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:57.443575    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:55:59.483207 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:55:59.483166    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:56:14.816581 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:14.816498    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:56:14.847726 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:14.847697    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"]
Sep 04 23:56:14.847960 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:14.847921    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj" podUID="9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec" containerName="payload-pre-processing" containerID="cri-o://b77697b9c51be87d386ef4a19e943c2007ecde702a4e7a8117834e40584e45af" gracePeriod=30
Sep 04 23:56:14.874825 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:14.874788    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:56:14.930009 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:14.929980    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:56:15.028673 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.028629    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:56:15.087974 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.087949    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:56:15.104823 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.104804    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"
Sep 04 23:56:15.191834 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.191807    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4qg9\" (UniqueName: \"kubernetes.io/projected/9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec-kube-api-access-b4qg9\") pod \"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec\" (UID: \"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec\") "
Sep 04 23:56:15.192017 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.191863    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec-plugins-config-volume\") pod \"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec\" (UID: \"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec\") "
Sep 04 23:56:15.192259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.192236    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec" (UID: "9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:56:15.194090 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.194063    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec-kube-api-access-b4qg9" (OuterVolumeSpecName: "kube-api-access-b4qg9") pod "9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec" (UID: "9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec"). InnerVolumeSpecName "kube-api-access-b4qg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:56:15.292728 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.292703    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4qg9\" (UniqueName: \"kubernetes.io/projected/9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec-kube-api-access-b4qg9\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:56:15.292728 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.292727    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:56:15.767013 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.766975    2564 generic.go:356] "Generic (PLEG): container finished" podID="9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec" containerID="b77697b9c51be87d386ef4a19e943c2007ecde702a4e7a8117834e40584e45af" exitCode=0
Sep 04 23:56:15.767207 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.767036    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"
Sep 04 23:56:15.767207 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.767065    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj" event={"ID":"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec","Type":"ContainerDied","Data":"b77697b9c51be87d386ef4a19e943c2007ecde702a4e7a8117834e40584e45af"}
Sep 04 23:56:15.767207 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.767132    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj" event={"ID":"9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec","Type":"ContainerDied","Data":"93f26ea70e2daf51b2267e57e30d0166ec53c58aeaf204dbf0269b13fa51b5d1"}
Sep 04 23:56:15.767207 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.767163    2564 scope.go:117] "RemoveContainer" containerID="b77697b9c51be87d386ef4a19e943c2007ecde702a4e7a8117834e40584e45af"
Sep 04 23:56:15.776751 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.776733    2564 scope.go:117] "RemoveContainer" containerID="b77697b9c51be87d386ef4a19e943c2007ecde702a4e7a8117834e40584e45af"
Sep 04 23:56:15.777092 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:56:15.777074    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b77697b9c51be87d386ef4a19e943c2007ecde702a4e7a8117834e40584e45af\": container with ID starting with b77697b9c51be87d386ef4a19e943c2007ecde702a4e7a8117834e40584e45af not found: ID does not exist" containerID="b77697b9c51be87d386ef4a19e943c2007ecde702a4e7a8117834e40584e45af"
Sep 04 23:56:15.777176 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.777104    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b77697b9c51be87d386ef4a19e943c2007ecde702a4e7a8117834e40584e45af"} err="failed to get container status \"b77697b9c51be87d386ef4a19e943c2007ecde702a4e7a8117834e40584e45af\": rpc error: code = NotFound desc = could not find container \"b77697b9c51be87d386ef4a19e943c2007ecde702a4e7a8117834e40584e45af\": container with ID starting with b77697b9c51be87d386ef4a19e943c2007ecde702a4e7a8117834e40584e45af not found: ID does not exist"
Sep 04 23:56:15.788492 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.788452    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"]
Sep 04 23:56:15.791638 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:15.791618    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ede0daf1-55b7487d66-hskgj"]
Sep 04 23:56:16.174951 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:16.174923    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec" path="/var/lib/kubelet/pods/9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec/volumes"
Sep 04 23:56:18.108313 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:18.108272    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:56:23.092695 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:23.092666    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"]
Sep 04 23:56:23.189210 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:23.189175    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:56:38.112286 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.112227    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz" podUID="225f09a3-37de-478a-a216-1d6560948fad" containerName="main" containerID="cri-o://03a20079a7f3e1da69fa8a0cd1a045357c7539a0f59b41d304911c2bb81df720" gracePeriod=45
Sep 04 23:56:38.352524 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.352503    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:56:38.486575 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.486515    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/225f09a3-37de-478a-a216-1d6560948fad-tls-certs\") pod \"225f09a3-37de-478a-a216-1d6560948fad\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") "
Sep 04 23:56:38.486575 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.486574    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-dshm\") pod \"225f09a3-37de-478a-a216-1d6560948fad\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") "
Sep 04 23:56:38.486765 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.486747    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-model-cache\") pod \"225f09a3-37de-478a-a216-1d6560948fad\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") "
Sep 04 23:56:38.486859 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.486843    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-home\") pod \"225f09a3-37de-478a-a216-1d6560948fad\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") "
Sep 04 23:56:38.486965 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.486865    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-tmp-dir\") pod \"225f09a3-37de-478a-a216-1d6560948fad\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") "
Sep 04 23:56:38.486965 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.486885    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bd2lq\" (UniqueName: \"kubernetes.io/projected/225f09a3-37de-478a-a216-1d6560948fad-kube-api-access-bd2lq\") pod \"225f09a3-37de-478a-a216-1d6560948fad\" (UID: \"225f09a3-37de-478a-a216-1d6560948fad\") "
Sep 04 23:56:38.487076 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.487010    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-model-cache" (OuterVolumeSpecName: "model-cache") pod "225f09a3-37de-478a-a216-1d6560948fad" (UID: "225f09a3-37de-478a-a216-1d6560948fad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:56:38.487294 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.487085    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-home" (OuterVolumeSpecName: "home") pod "225f09a3-37de-478a-a216-1d6560948fad" (UID: "225f09a3-37de-478a-a216-1d6560948fad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:56:38.487294 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.487171    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "225f09a3-37de-478a-a216-1d6560948fad" (UID: "225f09a3-37de-478a-a216-1d6560948fad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:56:38.487294 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.487245    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-model-cache\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:56:38.487294 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.487259    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-home\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:56:38.488857 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.488824    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-dshm" (OuterVolumeSpecName: "dshm") pod "225f09a3-37de-478a-a216-1d6560948fad" (UID: "225f09a3-37de-478a-a216-1d6560948fad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:56:38.489011 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.488891    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/225f09a3-37de-478a-a216-1d6560948fad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "225f09a3-37de-478a-a216-1d6560948fad" (UID: "225f09a3-37de-478a-a216-1d6560948fad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:56:38.489011 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.488936    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/225f09a3-37de-478a-a216-1d6560948fad-kube-api-access-bd2lq" (OuterVolumeSpecName: "kube-api-access-bd2lq") pod "225f09a3-37de-478a-a216-1d6560948fad" (UID: "225f09a3-37de-478a-a216-1d6560948fad"). InnerVolumeSpecName "kube-api-access-bd2lq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:56:38.587796 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.587767    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-tmp-dir\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:56:38.587927 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.587799    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd2lq\" (UniqueName: \"kubernetes.io/projected/225f09a3-37de-478a-a216-1d6560948fad-kube-api-access-bd2lq\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:56:38.587927 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.587816    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/225f09a3-37de-478a-a216-1d6560948fad-tls-certs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:56:38.587927 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.587827    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/225f09a3-37de-478a-a216-1d6560948fad-dshm\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:56:38.884278 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.884247    2564 generic.go:356] "Generic (PLEG): container finished" podID="225f09a3-37de-478a-a216-1d6560948fad" containerID="03a20079a7f3e1da69fa8a0cd1a045357c7539a0f59b41d304911c2bb81df720" exitCode=0
Sep 04 23:56:38.884418 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.884295    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"
Sep 04 23:56:38.884418 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.884333    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz" event={"ID":"225f09a3-37de-478a-a216-1d6560948fad","Type":"ContainerDied","Data":"03a20079a7f3e1da69fa8a0cd1a045357c7539a0f59b41d304911c2bb81df720"}
Sep 04 23:56:38.884418 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.884385    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz" event={"ID":"225f09a3-37de-478a-a216-1d6560948fad","Type":"ContainerDied","Data":"004b993accef1f00110b1ef6b76d472cc845ccb8baccd215535054cbf33d0994"}
Sep 04 23:56:38.884418 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.884408    2564 scope.go:117] "RemoveContainer" containerID="03a20079a7f3e1da69fa8a0cd1a045357c7539a0f59b41d304911c2bb81df720"
Sep 04 23:56:38.893967 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.893945    2564 scope.go:117] "RemoveContainer" containerID="03a20079a7f3e1da69fa8a0cd1a045357c7539a0f59b41d304911c2bb81df720"
Sep 04 23:56:38.894263 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:56:38.894245    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03a20079a7f3e1da69fa8a0cd1a045357c7539a0f59b41d304911c2bb81df720\": container with ID starting with 03a20079a7f3e1da69fa8a0cd1a045357c7539a0f59b41d304911c2bb81df720 not found: ID does not exist" containerID="03a20079a7f3e1da69fa8a0cd1a045357c7539a0f59b41d304911c2bb81df720"
Sep 04 23:56:38.894328 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.894271    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a20079a7f3e1da69fa8a0cd1a045357c7539a0f59b41d304911c2bb81df720"} err="failed to get container status \"03a20079a7f3e1da69fa8a0cd1a045357c7539a0f59b41d304911c2bb81df720\": rpc error: code = NotFound desc = could not find container \"03a20079a7f3e1da69fa8a0cd1a045357c7539a0f59b41d304911c2bb81df720\": container with ID starting with 03a20079a7f3e1da69fa8a0cd1a045357c7539a0f59b41d304911c2bb81df720 not found: ID does not exist"
Sep 04 23:56:38.905286 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.905221    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"]
Sep 04 23:56:38.907433 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:38.907415    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ede0daf1/e2e-lifecycle-model-ede0daf1-kserve-74d9f84f47-jb8qz"]
Sep 04 23:56:40.175741 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:40.175708    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="225f09a3-37de-478a-a216-1d6560948fad" path="/var/lib/kubelet/pods/225f09a3-37de-478a-a216-1d6560948fad/volumes"
Sep 04 23:56:59.630051 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:56:59.630014    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:00.031519 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:00.031435    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:05.570441 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:05.570395    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:05.670629 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:05.670596    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:05.817433 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:05.817397    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:07.081801 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.081768    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:07.185927 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.185875    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:07.828838 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.828792    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:07.938501 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.938473    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7633-333b9d"]
Sep 04 23:57:07.938839 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.938828    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec" containerName="payload-pre-processing"
Sep 04 23:57:07.938883 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.938841    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec" containerName="payload-pre-processing"
Sep 04 23:57:07.938883 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.938850    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="225f09a3-37de-478a-a216-1d6560948fad" containerName="main"
Sep 04 23:57:07.938883 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.938856    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="225f09a3-37de-478a-a216-1d6560948fad" containerName="main"
Sep 04 23:57:07.939005 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.938969    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="225f09a3-37de-478a-a216-1d6560948fad" containerName="main"
Sep 04 23:57:07.939005 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.938978    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d6b6e8d-c6d8-408a-9020-37ea8a97a6ec" containerName="payload-pre-processing"
Sep 04 23:57:07.943314 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.943299    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7633-333b9d"
Sep 04 23:57:07.946030 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.946006    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5vfk6\""
Sep 04 23:57:07.948260 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.948241    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7633-333b9d"]
Sep 04 23:57:07.964261 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:07.964234    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:08.021743 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:08.021720    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqlwz\" (UniqueName: \"kubernetes.io/projected/8528e403-1eb6-4d09-8cbb-268f493fad62-kube-api-access-rqlwz\") pod \"mt-curl-7633-333b9d\" (UID: \"8528e403-1eb6-4d09-8cbb-268f493fad62\") " pod="opendatahub/mt-curl-7633-333b9d"
Sep 04 23:57:08.029180 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:08.029147    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:08.122538 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:08.122517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqlwz\" (UniqueName: \"kubernetes.io/projected/8528e403-1eb6-4d09-8cbb-268f493fad62-kube-api-access-rqlwz\") pod \"mt-curl-7633-333b9d\" (UID: \"8528e403-1eb6-4d09-8cbb-268f493fad62\") " pod="opendatahub/mt-curl-7633-333b9d"
Sep 04 23:57:08.130504 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:08.130476    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqlwz\" (UniqueName: \"kubernetes.io/projected/8528e403-1eb6-4d09-8cbb-268f493fad62-kube-api-access-rqlwz\") pod \"mt-curl-7633-333b9d\" (UID: \"8528e403-1eb6-4d09-8cbb-268f493fad62\") " pod="opendatahub/mt-curl-7633-333b9d"
Sep 04 23:57:08.253549 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:08.253523    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7633-333b9d"
Sep 04 23:57:08.520388 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:08.520356    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7633-333b9d"]
Sep 04 23:57:08.522956 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:57:08.522931    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8528e403_1eb6_4d09_8cbb_268f493fad62.slice/crio-34aef4da980364dec6fb5917476dbbda2160912dd087a19825bd89a4558005c9 WatchSource:0}: Error finding container 34aef4da980364dec6fb5917476dbbda2160912dd087a19825bd89a4558005c9: Status 404 returned error can't find the container with id 34aef4da980364dec6fb5917476dbbda2160912dd087a19825bd89a4558005c9
Sep 04 23:57:09.037718 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:09.037683    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7633-333b9d" event={"ID":"8528e403-1eb6-4d09-8cbb-268f493fad62","Type":"ContainerStarted","Data":"26ae469631216fb3e12c2267a5c16cb8c9fcc54f91913a6e86f1cf1fd2efe5f7"}
Sep 04 23:57:09.037718 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:09.037722    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7633-333b9d" event={"ID":"8528e403-1eb6-4d09-8cbb-268f493fad62","Type":"ContainerStarted","Data":"34aef4da980364dec6fb5917476dbbda2160912dd087a19825bd89a4558005c9"}
Sep 04 23:57:09.052175 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:09.052127    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/mt-curl-7633-333b9d" podStartSLOduration=2.052113917 podStartE2EDuration="2.052113917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:57:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:57:09.050492927 +0000 UTC m=+1217.298493446" watchObservedRunningTime="2026-09-04 23:57:09.052113917 +0000 UTC m=+1217.300114432"
Sep 04 23:57:09.238460 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:09.238431    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:09.605081 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:09.605044    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7633-333b9d"]
Sep 04 23:57:09.608855 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:09.608831    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7633-333b9d"]
Sep 04 23:57:10.042398 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:57:10.042362    2564 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/mt-curl-7633-333b9d" container="mt-curl-7633-333b9d"
Sep 04 23:57:11.047385 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:11.047347    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/mt-curl-7633-333b9d" podUID="8528e403-1eb6-4d09-8cbb-268f493fad62" containerName="mt-curl-7633-333b9d" containerID="cri-o://26ae469631216fb3e12c2267a5c16cb8c9fcc54f91913a6e86f1cf1fd2efe5f7" gracePeriod=2
Sep 04 23:57:11.321948 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:11.321842    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:11.813159 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:11.813125    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"]
Sep 04 23:57:11.825505 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:11.825479    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:13.224197 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:13.224173    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7633-333b9d"
Sep 04 23:57:13.265364 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:13.265336    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqlwz\" (UniqueName: \"kubernetes.io/projected/8528e403-1eb6-4d09-8cbb-268f493fad62-kube-api-access-rqlwz\") pod \"8528e403-1eb6-4d09-8cbb-268f493fad62\" (UID: \"8528e403-1eb6-4d09-8cbb-268f493fad62\") "
Sep 04 23:57:13.267542 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:13.267512    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8528e403-1eb6-4d09-8cbb-268f493fad62-kube-api-access-rqlwz" (OuterVolumeSpecName: "kube-api-access-rqlwz") pod "8528e403-1eb6-4d09-8cbb-268f493fad62" (UID: "8528e403-1eb6-4d09-8cbb-268f493fad62"). InnerVolumeSpecName "kube-api-access-rqlwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:57:13.366524 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:13.366488    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqlwz\" (UniqueName: \"kubernetes.io/projected/8528e403-1eb6-4d09-8cbb-268f493fad62-kube-api-access-rqlwz\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:57:14.063953 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:14.063914    2564 generic.go:356] "Generic (PLEG): container finished" podID="8528e403-1eb6-4d09-8cbb-268f493fad62" containerID="26ae469631216fb3e12c2267a5c16cb8c9fcc54f91913a6e86f1cf1fd2efe5f7" exitCode=137
Sep 04 23:57:14.064155 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:14.063972    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7633-333b9d"
Sep 04 23:57:14.064155 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:14.064014    2564 scope.go:117] "RemoveContainer" containerID="26ae469631216fb3e12c2267a5c16cb8c9fcc54f91913a6e86f1cf1fd2efe5f7"
Sep 04 23:57:14.074110 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:14.074092    2564 scope.go:117] "RemoveContainer" containerID="26ae469631216fb3e12c2267a5c16cb8c9fcc54f91913a6e86f1cf1fd2efe5f7"
Sep 04 23:57:14.074400 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:57:14.074381    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26ae469631216fb3e12c2267a5c16cb8c9fcc54f91913a6e86f1cf1fd2efe5f7\": container with ID starting with 26ae469631216fb3e12c2267a5c16cb8c9fcc54f91913a6e86f1cf1fd2efe5f7 not found: ID does not exist" containerID="26ae469631216fb3e12c2267a5c16cb8c9fcc54f91913a6e86f1cf1fd2efe5f7"
Sep 04 23:57:14.074477 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:14.074408    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ae469631216fb3e12c2267a5c16cb8c9fcc54f91913a6e86f1cf1fd2efe5f7"} err="failed to get container status \"26ae469631216fb3e12c2267a5c16cb8c9fcc54f91913a6e86f1cf1fd2efe5f7\": rpc error: code = NotFound desc = could not find container \"26ae469631216fb3e12c2267a5c16cb8c9fcc54f91913a6e86f1cf1fd2efe5f7\": container with ID starting with 26ae469631216fb3e12c2267a5c16cb8c9fcc54f91913a6e86f1cf1fd2efe5f7 not found: ID does not exist"
Sep 04 23:57:14.075126 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:57:14.075105    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7633-333b9d\" is forbidden: User \"system:node:ip-10-0-139-127.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-127.ec2.internal' and this object" podUID="8528e403-1eb6-4d09-8cbb-268f493fad62" pod="opendatahub/mt-curl-7633-333b9d"
Sep 04 23:57:14.175179 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:14.175151    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8528e403-1eb6-4d09-8cbb-268f493fad62" path="/var/lib/kubelet/pods/8528e403-1eb6-4d09-8cbb-268f493fad62/volumes"
Sep 04 23:57:15.026761 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:15.026723    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:15.127073 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:15.127037    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:15.724186 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:15.724145    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:21.745252 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:21.745219    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:21.842972 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:21.842936    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:26.833578 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:26.833505    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4" podUID="d23936cb-9c57-44d9-ab96-f3a7e3a98fc3" containerName="main" containerID="cri-o://c1a8ba7e7f615c62b4587e0c44878d9e21205e65f8350c975410ae1c355e6995" gracePeriod=45
Sep 04 23:57:27.087003 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.086943    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:57:27.128368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.128335    2564 generic.go:356] "Generic (PLEG): container finished" podID="d23936cb-9c57-44d9-ab96-f3a7e3a98fc3" containerID="c1a8ba7e7f615c62b4587e0c44878d9e21205e65f8350c975410ae1c355e6995" exitCode=0
Sep 04 23:57:27.128501 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.128395    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"
Sep 04 23:57:27.128501 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.128421    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4" event={"ID":"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3","Type":"ContainerDied","Data":"c1a8ba7e7f615c62b4587e0c44878d9e21205e65f8350c975410ae1c355e6995"}
Sep 04 23:57:27.128501 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.128470    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4" event={"ID":"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3","Type":"ContainerDied","Data":"62db3af87c6921952641d558a947029cd52458f94257e1e3fbf3c5374ce2080d"}
Sep 04 23:57:27.128501 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.128488    2564 scope.go:117] "RemoveContainer" containerID="c1a8ba7e7f615c62b4587e0c44878d9e21205e65f8350c975410ae1c355e6995"
Sep 04 23:57:27.137139 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.137122    2564 scope.go:117] "RemoveContainer" containerID="c1a8ba7e7f615c62b4587e0c44878d9e21205e65f8350c975410ae1c355e6995"
Sep 04 23:57:27.137404 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:57:27.137385    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1a8ba7e7f615c62b4587e0c44878d9e21205e65f8350c975410ae1c355e6995\": container with ID starting with c1a8ba7e7f615c62b4587e0c44878d9e21205e65f8350c975410ae1c355e6995 not found: ID does not exist" containerID="c1a8ba7e7f615c62b4587e0c44878d9e21205e65f8350c975410ae1c355e6995"
Sep 04 23:57:27.137465 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.137411    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1a8ba7e7f615c62b4587e0c44878d9e21205e65f8350c975410ae1c355e6995"} err="failed to get container status \"c1a8ba7e7f615c62b4587e0c44878d9e21205e65f8350c975410ae1c355e6995\": rpc error: code = NotFound desc = could not find container \"c1a8ba7e7f615c62b4587e0c44878d9e21205e65f8350c975410ae1c355e6995\": container with ID starting with c1a8ba7e7f615c62b4587e0c44878d9e21205e65f8350c975410ae1c355e6995 not found: ID does not exist"
Sep 04 23:57:27.191697 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.191670    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-home\") pod \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") "
Sep 04 23:57:27.191697 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.191701    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-model-cache\") pod \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") "
Sep 04 23:57:27.191883 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.191736    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-tmp-dir\") pod \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") "
Sep 04 23:57:27.191883 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.191837    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qgn6\" (UniqueName: \"kubernetes.io/projected/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-kube-api-access-6qgn6\") pod \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") "
Sep 04 23:57:27.191883 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.191854    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-tls-certs\") pod \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") "
Sep 04 23:57:27.192064 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.191893    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-dshm\") pod \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\" (UID: \"d23936cb-9c57-44d9-ab96-f3a7e3a98fc3\") "
Sep 04 23:57:27.192064 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.191999    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-home" (OuterVolumeSpecName: "home") pod "d23936cb-9c57-44d9-ab96-f3a7e3a98fc3" (UID: "d23936cb-9c57-44d9-ab96-f3a7e3a98fc3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:57:27.192064 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.192009    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-model-cache" (OuterVolumeSpecName: "model-cache") pod "d23936cb-9c57-44d9-ab96-f3a7e3a98fc3" (UID: "d23936cb-9c57-44d9-ab96-f3a7e3a98fc3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:57:27.192219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.192093    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d23936cb-9c57-44d9-ab96-f3a7e3a98fc3" (UID: "d23936cb-9c57-44d9-ab96-f3a7e3a98fc3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:57:27.192219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.192180    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-home\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:57:27.192219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.192197    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-model-cache\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:57:27.192219 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.192212    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-tmp-dir\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:57:27.194268 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.194245    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-dshm" (OuterVolumeSpecName: "dshm") pod "d23936cb-9c57-44d9-ab96-f3a7e3a98fc3" (UID: "d23936cb-9c57-44d9-ab96-f3a7e3a98fc3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:57:27.194343 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.194308    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d23936cb-9c57-44d9-ab96-f3a7e3a98fc3" (UID: "d23936cb-9c57-44d9-ab96-f3a7e3a98fc3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:57:27.194390 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.194362    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-kube-api-access-6qgn6" (OuterVolumeSpecName: "kube-api-access-6qgn6") pod "d23936cb-9c57-44d9-ab96-f3a7e3a98fc3" (UID: "d23936cb-9c57-44d9-ab96-f3a7e3a98fc3"). InnerVolumeSpecName "kube-api-access-6qgn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:57:27.293636 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.293601    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qgn6\" (UniqueName: \"kubernetes.io/projected/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-kube-api-access-6qgn6\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:57:27.293636 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.293632    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-tls-certs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:57:27.293636 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.293642    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3-dshm\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:57:27.449399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.449369    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"]
Sep 04 23:57:27.452383 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.452357    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-88828d/auth-test-model-88828d-kserve-76cfd49bb7-nmjk4"]
Sep 04 23:57:27.622739 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.622711    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"]
Sep 04 23:57:27.623119 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.623105    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8528e403-1eb6-4d09-8cbb-268f493fad62" containerName="mt-curl-7633-333b9d"
Sep 04 23:57:27.623182 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.623121    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8528e403-1eb6-4d09-8cbb-268f493fad62" containerName="mt-curl-7633-333b9d"
Sep 04 23:57:27.623182 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.623131    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23936cb-9c57-44d9-ab96-f3a7e3a98fc3" containerName="main"
Sep 04 23:57:27.623182 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.623137    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23936cb-9c57-44d9-ab96-f3a7e3a98fc3" containerName="main"
Sep 04 23:57:27.623291 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.623201    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23936cb-9c57-44d9-ab96-f3a7e3a98fc3" containerName="main"
Sep 04 23:57:27.623291 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.623210    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8528e403-1eb6-4d09-8cbb-268f493fad62" containerName="mt-curl-7633-333b9d"
Sep 04 23:57:27.627544 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.627514    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.630085 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.630061    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5245c1\"/\"sub-test-model-5245c1-kserve-self-signed-certs\""
Sep 04 23:57:27.630085 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.630074    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5245c1\"/\"openshift-service-ca.crt\""
Sep 04 23:57:27.630215 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.630075    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5245c1\"/\"default-dockercfg-bdmqt\""
Sep 04 23:57:27.630215 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.630074    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5245c1\"/\"kube-root-ca.crt\""
Sep 04 23:57:27.634346 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.634320    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"]
Sep 04 23:57:27.647062 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.647024    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:27.697263 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.697188    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/946c686c-9a43-4442-9d88-c24f66d0bf2a-tls-certs\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.697263 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.697225    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-tmp-dir\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.697263 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.697246    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlwn7\" (UniqueName: \"kubernetes.io/projected/946c686c-9a43-4442-9d88-c24f66d0bf2a-kube-api-access-jlwn7\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.697458 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.697327    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-dshm\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.697458 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.697422    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-model-cache\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.697458 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.697454    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-home\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.798586 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.798544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/946c686c-9a43-4442-9d88-c24f66d0bf2a-tls-certs\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.798586 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.798589    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-tmp-dir\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.798823 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.798608    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlwn7\" (UniqueName: \"kubernetes.io/projected/946c686c-9a43-4442-9d88-c24f66d0bf2a-kube-api-access-jlwn7\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.798823 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.798633    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-dshm\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.798823 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.798674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-model-cache\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.798823 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.798692    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-home\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.799104 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.799080    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-model-cache\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.799161 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.799089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-tmp-dir\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.799161 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.799144    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-home\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.801026 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.801005    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-dshm\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.801320 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.801302    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/946c686c-9a43-4442-9d88-c24f66d0bf2a-tls-certs\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.805911 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.805882    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlwn7\" (UniqueName: \"kubernetes.io/projected/946c686c-9a43-4442-9d88-c24f66d0bf2a-kube-api-access-jlwn7\") pod \"sub-test-model-5245c1-kserve-76dc98694c-cbng6\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:27.937818 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:27.937779    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:28.175028 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:28.174988    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d23936cb-9c57-44d9-ab96-f3a7e3a98fc3" path="/var/lib/kubelet/pods/d23936cb-9c57-44d9-ab96-f3a7e3a98fc3/volumes"
Sep 04 23:57:28.209667 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:28.209641    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"]
Sep 04 23:57:28.212164 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:57:28.212140    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod946c686c_9a43_4442_9d88_c24f66d0bf2a.slice/crio-a8af51928a957748d754cb88962a82b613578aec6516dca3f270157a45aaaf75 WatchSource:0}: Error finding container a8af51928a957748d754cb88962a82b613578aec6516dca3f270157a45aaaf75: Status 404 returned error can't find the container with id a8af51928a957748d754cb88962a82b613578aec6516dca3f270157a45aaaf75
Sep 04 23:57:29.146860 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:29.146828    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6" event={"ID":"946c686c-9a43-4442-9d88-c24f66d0bf2a","Type":"ContainerStarted","Data":"879f5672ef35e29504b0f3326bb5e6a1c1755937cb3dbacb29ade44050704854"}
Sep 04 23:57:29.147235 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:29.146872    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6" event={"ID":"946c686c-9a43-4442-9d88-c24f66d0bf2a","Type":"ContainerStarted","Data":"a8af51928a957748d754cb88962a82b613578aec6516dca3f270157a45aaaf75"}
Sep 04 23:57:29.163908 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:29.163855    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6" podStartSLOduration=2.163840512 podStartE2EDuration="2.163840512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:57:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:57:29.161673258 +0000 UTC m=+1237.409673795" watchObservedRunningTime="2026-09-04 23:57:29.163840512 +0000 UTC m=+1237.411841027"
Sep 04 23:57:32.229562 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:32.229523    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:32.782398 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:32.782361    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:34.072028 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:34.071993    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:37.938368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:37.938334    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:37.940757 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:37.938383    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:37.950638 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:37.950616    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:38.005074 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:38.005048    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:38.204533 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:38.204461    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:57:53.421423 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:53.421381    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:55.552892 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:55.552855    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:59.242736 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:59.242697    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:59.339463 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:59.339430    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:59.608352 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:59.608326    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:57:59.968572 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:57:59.968464    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:00.138485 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:00.138450    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:02.940776 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:02.940738    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:03.042100 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:03.042069    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:04.069487 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.069408    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:04.169243 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.169207    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:04.392083 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.392055    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:04.553159 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.553123    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:04.652914 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.652816    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:04.658798 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.658773    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"]
Sep 04 23:58:04.664113 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.664095    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"
Sep 04 23:58:04.666685 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.666656    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-87b0d1\""
Sep 04 23:58:04.667068 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.667052    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-87b0d1-dockercfg-fklww\""
Sep 04 23:58:04.672889 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.672870    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"]
Sep 04 23:58:04.688065 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.688045    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"]
Sep 04 23:58:04.691479 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.691461    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"
Sep 04 23:58:04.700322 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.700300    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"]
Sep 04 23:58:04.720606 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.720580    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/677b6701-ac1e-4d07-b385-9f7953e3cb92-plugins-config-volume\") pod \"payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j\" (UID: \"677b6701-ac1e-4d07-b385-9f7953e3cb92\") " pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"
Sep 04 23:58:04.720708 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.720611    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62vh6\" (UniqueName: \"kubernetes.io/projected/b3b33412-37db-4ba0-a455-0fbedd26925d-kube-api-access-62vh6\") pod \"payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8\" (UID: \"b3b33412-37db-4ba0-a455-0fbedd26925d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"
Sep 04 23:58:04.720708 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.720655    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3b33412-37db-4ba0-a455-0fbedd26925d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8\" (UID: \"b3b33412-37db-4ba0-a455-0fbedd26925d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"
Sep 04 23:58:04.720825 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.720756    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8kdp\" (UniqueName: \"kubernetes.io/projected/677b6701-ac1e-4d07-b385-9f7953e3cb92-kube-api-access-c8kdp\") pod \"payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j\" (UID: \"677b6701-ac1e-4d07-b385-9f7953e3cb92\") " pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"
Sep 04 23:58:04.821456 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.821431    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/677b6701-ac1e-4d07-b385-9f7953e3cb92-plugins-config-volume\") pod \"payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j\" (UID: \"677b6701-ac1e-4d07-b385-9f7953e3cb92\") " pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"
Sep 04 23:58:04.821568 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.821463    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62vh6\" (UniqueName: \"kubernetes.io/projected/b3b33412-37db-4ba0-a455-0fbedd26925d-kube-api-access-62vh6\") pod \"payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8\" (UID: \"b3b33412-37db-4ba0-a455-0fbedd26925d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"
Sep 04 23:58:04.821568 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.821504    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3b33412-37db-4ba0-a455-0fbedd26925d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8\" (UID: \"b3b33412-37db-4ba0-a455-0fbedd26925d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"
Sep 04 23:58:04.821568 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.821530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8kdp\" (UniqueName: \"kubernetes.io/projected/677b6701-ac1e-4d07-b385-9f7953e3cb92-kube-api-access-c8kdp\") pod \"payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j\" (UID: \"677b6701-ac1e-4d07-b385-9f7953e3cb92\") " pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"
Sep 04 23:58:04.822216 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.822194    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/677b6701-ac1e-4d07-b385-9f7953e3cb92-plugins-config-volume\") pod \"payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j\" (UID: \"677b6701-ac1e-4d07-b385-9f7953e3cb92\") " pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"
Sep 04 23:58:04.822300 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.822264    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3b33412-37db-4ba0-a455-0fbedd26925d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8\" (UID: \"b3b33412-37db-4ba0-a455-0fbedd26925d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"
Sep 04 23:58:04.829387 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.829367    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62vh6\" (UniqueName: \"kubernetes.io/projected/b3b33412-37db-4ba0-a455-0fbedd26925d-kube-api-access-62vh6\") pod \"payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8\" (UID: \"b3b33412-37db-4ba0-a455-0fbedd26925d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"
Sep 04 23:58:04.829930 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.829881    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8kdp\" (UniqueName: \"kubernetes.io/projected/677b6701-ac1e-4d07-b385-9f7953e3cb92-kube-api-access-c8kdp\") pod \"payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j\" (UID: \"677b6701-ac1e-4d07-b385-9f7953e3cb92\") " pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"
Sep 04 23:58:04.976105 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:04.976053    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"
Sep 04 23:58:05.002091 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:05.002067    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"
Sep 04 23:58:05.249558 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:05.249537    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"]
Sep 04 23:58:05.251916 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:58:05.251873    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3b33412_37db_4ba0_a455_0fbedd26925d.slice/crio-358ac5804d4473e015f87c07a87806042982133a4409f461747f04f2435d620a WatchSource:0}: Error finding container 358ac5804d4473e015f87c07a87806042982133a4409f461747f04f2435d620a: Status 404 returned error can't find the container with id 358ac5804d4473e015f87c07a87806042982133a4409f461747f04f2435d620a
Sep 04 23:58:05.280584 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:05.280544    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"]
Sep 04 23:58:05.282145 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:58:05.282124    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod677b6701_ac1e_4d07_b385_9f7953e3cb92.slice/crio-dd2594e050ecc916d130e0c8006b959f955d6b02eb2be7e1ea6ec958be2cbf4b WatchSource:0}: Error finding container dd2594e050ecc916d130e0c8006b959f955d6b02eb2be7e1ea6ec958be2cbf4b: Status 404 returned error can't find the container with id dd2594e050ecc916d130e0c8006b959f955d6b02eb2be7e1ea6ec958be2cbf4b
Sep 04 23:58:05.324723 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:05.324700    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8" event={"ID":"b3b33412-37db-4ba0-a455-0fbedd26925d","Type":"ContainerStarted","Data":"358ac5804d4473e015f87c07a87806042982133a4409f461747f04f2435d620a"}
Sep 04 23:58:05.325960 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:05.325937    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j" event={"ID":"677b6701-ac1e-4d07-b385-9f7953e3cb92","Type":"ContainerStarted","Data":"dd2594e050ecc916d130e0c8006b959f955d6b02eb2be7e1ea6ec958be2cbf4b"}
Sep 04 23:58:06.332449 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:06.332413    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j" event={"ID":"677b6701-ac1e-4d07-b385-9f7953e3cb92","Type":"ContainerStarted","Data":"6fe44b1fd9e672bc8cd9f166895dff7fdcdaff30308950108848eedeab3d7ed1"}
Sep 04 23:58:06.332847 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:06.332567    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"
Sep 04 23:58:06.337108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:06.337078    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8" event={"ID":"b3b33412-37db-4ba0-a455-0fbedd26925d","Type":"ContainerStarted","Data":"2c25601dd9ffea048bb3181de03af1f25efce96285c5d7bf38f0785c7003fdb6"}
Sep 04 23:58:06.337245 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:06.337134    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"
Sep 04 23:58:06.350112 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:06.350066    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j" podStartSLOduration=2.350052304 podStartE2EDuration="2.350052304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:58:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:58:06.348440075 +0000 UTC m=+1274.596440615" watchObservedRunningTime="2026-09-04 23:58:06.350052304 +0000 UTC m=+1274.598052814"
Sep 04 23:58:06.363165 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:06.363117    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8" podStartSLOduration=2.363104389 podStartE2EDuration="2.363104389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:58:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:58:06.362423677 +0000 UTC m=+1274.610424219" watchObservedRunningTime="2026-09-04 23:58:06.363104389 +0000 UTC m=+1274.611104904"
Sep 04 23:58:17.344466 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:17.344435    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"
Sep 04 23:58:17.345091 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:17.344652    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"
Sep 04 23:58:27.876767 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:27.876721    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:28.393300 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:28.393264    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:28.586863 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:28.586827    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:28.685668 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:28.685588    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:31.202076 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:31.202047    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:31.300813 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:31.300780    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:32.665371 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:32.665340    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:32.761623 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:32.761585    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:32.917482 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:32.917402    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:33.012207 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:33.012169    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:33.319680 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:33.319587    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:34.250918 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:34.250864    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7633-6407ec"]
Sep 04 23:58:34.254664 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:34.254641    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7633-6407ec"
Sep 04 23:58:34.257198 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:34.257176    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5vfk6\""
Sep 04 23:58:34.258046 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:34.258020    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7633-6407ec"]
Sep 04 23:58:34.371343 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:34.371316    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg5l4\" (UniqueName: \"kubernetes.io/projected/4e5ee98f-0593-49fa-b989-f09a3a6e27f2-kube-api-access-cg5l4\") pod \"mt-curl-7633-6407ec\" (UID: \"4e5ee98f-0593-49fa-b989-f09a3a6e27f2\") " pod="opendatahub/mt-curl-7633-6407ec"
Sep 04 23:58:34.472538 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:34.472514    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg5l4\" (UniqueName: \"kubernetes.io/projected/4e5ee98f-0593-49fa-b989-f09a3a6e27f2-kube-api-access-cg5l4\") pod \"mt-curl-7633-6407ec\" (UID: \"4e5ee98f-0593-49fa-b989-f09a3a6e27f2\") " pod="opendatahub/mt-curl-7633-6407ec"
Sep 04 23:58:34.488807 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:34.488777    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg5l4\" (UniqueName: \"kubernetes.io/projected/4e5ee98f-0593-49fa-b989-f09a3a6e27f2-kube-api-access-cg5l4\") pod \"mt-curl-7633-6407ec\" (UID: \"4e5ee98f-0593-49fa-b989-f09a3a6e27f2\") " pod="opendatahub/mt-curl-7633-6407ec"
Sep 04 23:58:34.567536 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:34.567472    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7633-6407ec"
Sep 04 23:58:34.833828 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:34.833805    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7633-6407ec"]
Sep 04 23:58:34.835845 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:58:34.835822    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e5ee98f_0593_49fa_b989_f09a3a6e27f2.slice/crio-d673d9b1643406364c2f4cbe87ed85b84bd70efa65f05d978b52a370f1e8bed8 WatchSource:0}: Error finding container d673d9b1643406364c2f4cbe87ed85b84bd70efa65f05d978b52a370f1e8bed8: Status 404 returned error can't find the container with id d673d9b1643406364c2f4cbe87ed85b84bd70efa65f05d978b52a370f1e8bed8
Sep 04 23:58:35.489003 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:35.488960    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7633-6407ec" event={"ID":"4e5ee98f-0593-49fa-b989-f09a3a6e27f2","Type":"ContainerStarted","Data":"c10acd2cdf536af36fa11a593c1888a2c7348414385e88472dc527e49343f24e"}
Sep 04 23:58:35.489003 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:35.489010    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7633-6407ec" event={"ID":"4e5ee98f-0593-49fa-b989-f09a3a6e27f2","Type":"ContainerStarted","Data":"d673d9b1643406364c2f4cbe87ed85b84bd70efa65f05d978b52a370f1e8bed8"}
Sep 04 23:58:35.502230 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:35.502185    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/mt-curl-7633-6407ec" podStartSLOduration=1.502170786 podStartE2EDuration="1.502170786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:58:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:58:35.50158 +0000 UTC m=+1303.749580517" watchObservedRunningTime="2026-09-04 23:58:35.502170786 +0000 UTC m=+1303.750171302"
Sep 04 23:58:36.074551 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:36.074477    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7633-6407ec"]
Sep 04 23:58:36.076856 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:36.076832    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7633-6407ec"]
Sep 04 23:58:36.294315 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:36.294246    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7633-f25b97"]
Sep 04 23:58:36.294665 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:36.294650    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e5ee98f-0593-49fa-b989-f09a3a6e27f2" containerName="mt-curl-7633-6407ec"
Sep 04 23:58:36.294708 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:36.294669    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e5ee98f-0593-49fa-b989-f09a3a6e27f2" containerName="mt-curl-7633-6407ec"
Sep 04 23:58:36.294754 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:36.294744    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e5ee98f-0593-49fa-b989-f09a3a6e27f2" containerName="mt-curl-7633-6407ec"
Sep 04 23:58:36.298065 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:36.298039    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7633-f25b97"
Sep 04 23:58:36.302260 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:36.302236    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7633-f25b97"]
Sep 04 23:58:36.388724 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:36.388700    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwh6t\" (UniqueName: \"kubernetes.io/projected/9ef73b28-e482-4dc9-849f-e0bf13a02636-kube-api-access-zwh6t\") pod \"mt-curl-7633-f25b97\" (UID: \"9ef73b28-e482-4dc9-849f-e0bf13a02636\") " pod="opendatahub/mt-curl-7633-f25b97"
Sep 04 23:58:36.489605 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:36.489578    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwh6t\" (UniqueName: \"kubernetes.io/projected/9ef73b28-e482-4dc9-849f-e0bf13a02636-kube-api-access-zwh6t\") pod \"mt-curl-7633-f25b97\" (UID: \"9ef73b28-e482-4dc9-849f-e0bf13a02636\") " pod="opendatahub/mt-curl-7633-f25b97"
Sep 04 23:58:36.493639 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:58:36.493607    2564 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/mt-curl-7633-6407ec" container="mt-curl-7633-6407ec"
Sep 04 23:58:36.498389 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:36.498368    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwh6t\" (UniqueName: \"kubernetes.io/projected/9ef73b28-e482-4dc9-849f-e0bf13a02636-kube-api-access-zwh6t\") pod \"mt-curl-7633-f25b97\" (UID: \"9ef73b28-e482-4dc9-849f-e0bf13a02636\") " pod="opendatahub/mt-curl-7633-f25b97"
Sep 04 23:58:36.608212 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:36.608195    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7633-f25b97"
Sep 04 23:58:36.874341 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:36.874281    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7633-f25b97"]
Sep 04 23:58:36.876717 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:58:36.876690    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ef73b28_e482_4dc9_849f_e0bf13a02636.slice/crio-29d20086dba77ed64f6449bae746fd966b12ede27a071d6670d6247b425a276e WatchSource:0}: Error finding container 29d20086dba77ed64f6449bae746fd966b12ede27a071d6670d6247b425a276e: Status 404 returned error can't find the container with id 29d20086dba77ed64f6449bae746fd966b12ede27a071d6670d6247b425a276e
Sep 04 23:58:37.499681 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:37.499650    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7633-f25b97" event={"ID":"9ef73b28-e482-4dc9-849f-e0bf13a02636","Type":"ContainerStarted","Data":"ed7550504d1cbe8c67397fc2830170e2da2f3b1a08ee7c01d9ae695540336184"}
Sep 04 23:58:37.500133 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:37.499692    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7633-f25b97" event={"ID":"9ef73b28-e482-4dc9-849f-e0bf13a02636","Type":"ContainerStarted","Data":"29d20086dba77ed64f6449bae746fd966b12ede27a071d6670d6247b425a276e"}
Sep 04 23:58:37.500133 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:37.499799    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/mt-curl-7633-6407ec" podUID="4e5ee98f-0593-49fa-b989-f09a3a6e27f2" containerName="mt-curl-7633-6407ec" containerID="cri-o://c10acd2cdf536af36fa11a593c1888a2c7348414385e88472dc527e49343f24e" gracePeriod=2
Sep 04 23:58:37.515241 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:37.515199    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/mt-curl-7633-f25b97" podStartSLOduration=1.515185013 podStartE2EDuration="1.515185013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:58:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:58:37.512432252 +0000 UTC m=+1305.760432779" watchObservedRunningTime="2026-09-04 23:58:37.515185013 +0000 UTC m=+1305.763185569"
Sep 04 23:58:38.073368 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:38.073331    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7633-f25b97"]
Sep 04 23:58:38.079391 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:38.079362    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7633-f25b97"]
Sep 04 23:58:38.505240 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:58:38.505209    2564 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/mt-curl-7633-f25b97" container="mt-curl-7633-f25b97"
Sep 04 23:58:39.511259 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:39.511224    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/mt-curl-7633-f25b97" podUID="9ef73b28-e482-4dc9-849f-e0bf13a02636" containerName="mt-curl-7633-f25b97" containerID="cri-o://ed7550504d1cbe8c67397fc2830170e2da2f3b1a08ee7c01d9ae695540336184" gracePeriod=2
Sep 04 23:58:39.675937 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:39.675893    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7633-6407ec"
Sep 04 23:58:39.818451 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:39.818388    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cg5l4\" (UniqueName: \"kubernetes.io/projected/4e5ee98f-0593-49fa-b989-f09a3a6e27f2-kube-api-access-cg5l4\") pod \"4e5ee98f-0593-49fa-b989-f09a3a6e27f2\" (UID: \"4e5ee98f-0593-49fa-b989-f09a3a6e27f2\") "
Sep 04 23:58:39.820434 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:39.820412    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e5ee98f-0593-49fa-b989-f09a3a6e27f2-kube-api-access-cg5l4" (OuterVolumeSpecName: "kube-api-access-cg5l4") pod "4e5ee98f-0593-49fa-b989-f09a3a6e27f2" (UID: "4e5ee98f-0593-49fa-b989-f09a3a6e27f2"). InnerVolumeSpecName "kube-api-access-cg5l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:58:39.919280 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:39.919255    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg5l4\" (UniqueName: \"kubernetes.io/projected/4e5ee98f-0593-49fa-b989-f09a3a6e27f2-kube-api-access-cg5l4\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:58:40.024547 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:40.024521    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"]
Sep 04 23:58:40.041718 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:40.041690    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:40.175648 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:40.175620    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e5ee98f-0593-49fa-b989-f09a3a6e27f2" path="/var/lib/kubelet/pods/4e5ee98f-0593-49fa-b989-f09a3a6e27f2/volumes"
Sep 04 23:58:40.516676 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:40.516604    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e5ee98f-0593-49fa-b989-f09a3a6e27f2" containerID="c10acd2cdf536af36fa11a593c1888a2c7348414385e88472dc527e49343f24e" exitCode=137
Sep 04 23:58:40.516676 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:40.516645    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7633-6407ec"
Sep 04 23:58:40.517127 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:40.516651    2564 scope.go:117] "RemoveContainer" containerID="c10acd2cdf536af36fa11a593c1888a2c7348414385e88472dc527e49343f24e"
Sep 04 23:58:40.525005 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:40.524981    2564 scope.go:117] "RemoveContainer" containerID="c10acd2cdf536af36fa11a593c1888a2c7348414385e88472dc527e49343f24e"
Sep 04 23:58:40.525256 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:58:40.525237    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c10acd2cdf536af36fa11a593c1888a2c7348414385e88472dc527e49343f24e\": container with ID starting with c10acd2cdf536af36fa11a593c1888a2c7348414385e88472dc527e49343f24e not found: ID does not exist" containerID="c10acd2cdf536af36fa11a593c1888a2c7348414385e88472dc527e49343f24e"
Sep 04 23:58:40.525325 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:40.525269    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c10acd2cdf536af36fa11a593c1888a2c7348414385e88472dc527e49343f24e"} err="failed to get container status \"c10acd2cdf536af36fa11a593c1888a2c7348414385e88472dc527e49343f24e\": rpc error: code = NotFound desc = could not find container \"c10acd2cdf536af36fa11a593c1888a2c7348414385e88472dc527e49343f24e\": container with ID starting with c10acd2cdf536af36fa11a593c1888a2c7348414385e88472dc527e49343f24e not found: ID does not exist"
Sep 04 23:58:41.020101 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:41.020070    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:41.694979 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:41.694953    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7633-f25b97"
Sep 04 23:58:41.835302 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:41.835230    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwh6t\" (UniqueName: \"kubernetes.io/projected/9ef73b28-e482-4dc9-849f-e0bf13a02636-kube-api-access-zwh6t\") pod \"9ef73b28-e482-4dc9-849f-e0bf13a02636\" (UID: \"9ef73b28-e482-4dc9-849f-e0bf13a02636\") "
Sep 04 23:58:41.837449 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:41.837421    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ef73b28-e482-4dc9-849f-e0bf13a02636-kube-api-access-zwh6t" (OuterVolumeSpecName: "kube-api-access-zwh6t") pod "9ef73b28-e482-4dc9-849f-e0bf13a02636" (UID: "9ef73b28-e482-4dc9-849f-e0bf13a02636"). InnerVolumeSpecName "kube-api-access-zwh6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:58:41.936493 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:41.936465    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwh6t\" (UniqueName: \"kubernetes.io/projected/9ef73b28-e482-4dc9-849f-e0bf13a02636-kube-api-access-zwh6t\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:58:42.174734 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:42.174704    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ef73b28-e482-4dc9-849f-e0bf13a02636" path="/var/lib/kubelet/pods/9ef73b28-e482-4dc9-849f-e0bf13a02636/volumes"
Sep 04 23:58:42.228025 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:42.227996    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:42.529424 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:42.529351    2564 generic.go:356] "Generic (PLEG): container finished" podID="9ef73b28-e482-4dc9-849f-e0bf13a02636" containerID="ed7550504d1cbe8c67397fc2830170e2da2f3b1a08ee7c01d9ae695540336184" exitCode=137
Sep 04 23:58:42.529424 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:42.529393    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7633-f25b97"
Sep 04 23:58:42.529608 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:42.529450    2564 scope.go:117] "RemoveContainer" containerID="ed7550504d1cbe8c67397fc2830170e2da2f3b1a08ee7c01d9ae695540336184"
Sep 04 23:58:42.537975 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:42.537955    2564 scope.go:117] "RemoveContainer" containerID="ed7550504d1cbe8c67397fc2830170e2da2f3b1a08ee7c01d9ae695540336184"
Sep 04 23:58:42.538270 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:58:42.538253    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed7550504d1cbe8c67397fc2830170e2da2f3b1a08ee7c01d9ae695540336184\": container with ID starting with ed7550504d1cbe8c67397fc2830170e2da2f3b1a08ee7c01d9ae695540336184 not found: ID does not exist" containerID="ed7550504d1cbe8c67397fc2830170e2da2f3b1a08ee7c01d9ae695540336184"
Sep 04 23:58:42.538385 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:42.538277    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed7550504d1cbe8c67397fc2830170e2da2f3b1a08ee7c01d9ae695540336184"} err="failed to get container status \"ed7550504d1cbe8c67397fc2830170e2da2f3b1a08ee7c01d9ae695540336184\": rpc error: code = NotFound desc = could not find container \"ed7550504d1cbe8c67397fc2830170e2da2f3b1a08ee7c01d9ae695540336184\": container with ID starting with ed7550504d1cbe8c67397fc2830170e2da2f3b1a08ee7c01d9ae695540336184 not found: ID does not exist"
Sep 04 23:58:50.013946 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.013916    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"]
Sep 04 23:58:50.014486 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.014389    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef73b28-e482-4dc9-849f-e0bf13a02636" containerName="mt-curl-7633-f25b97"
Sep 04 23:58:50.014486 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.014406    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef73b28-e482-4dc9-849f-e0bf13a02636" containerName="mt-curl-7633-f25b97"
Sep 04 23:58:50.014598 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.014491    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ef73b28-e482-4dc9-849f-e0bf13a02636" containerName="mt-curl-7633-f25b97"
Sep 04 23:58:50.018826 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.018806    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.021312 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.021275    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5245c1\"/\"rate-test-model-5245c1-kserve-self-signed-certs\""
Sep 04 23:58:50.026108 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.026088    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"]
Sep 04 23:58:50.040286 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.040260    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:50.203309 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.203283    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ef599b-cb26-42aa-b008-2f317bdf2b59-tls-certs\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.203422 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.203312    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-tmp-dir\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.203422 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.203332    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-dshm\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.203422 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.203364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-home\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.203422 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.203402    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-model-cache\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.203559 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.203428    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx6mb\" (UniqueName: \"kubernetes.io/projected/92ef599b-cb26-42aa-b008-2f317bdf2b59-kube-api-access-dx6mb\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.304079 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.304011    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ef599b-cb26-42aa-b008-2f317bdf2b59-tls-certs\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.304079 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.304049    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-tmp-dir\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.304079 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.304076    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-dshm\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.304293 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.304119    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-home\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.304293 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.304151    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-model-cache\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.304293 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.304182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx6mb\" (UniqueName: \"kubernetes.io/projected/92ef599b-cb26-42aa-b008-2f317bdf2b59-kube-api-access-dx6mb\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.304455 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.304433    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-tmp-dir\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.304519 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.304487    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-home\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.304576 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.304543    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-model-cache\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.306474 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.306456    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-dshm\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.306627 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.306605    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ef599b-cb26-42aa-b008-2f317bdf2b59-tls-certs\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.312172 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.312152    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx6mb\" (UniqueName: \"kubernetes.io/projected/92ef599b-cb26-42aa-b008-2f317bdf2b59-kube-api-access-dx6mb\") pod \"rate-test-model-5245c1-kserve-bcf4967b7-hnqtl\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") " pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.328666 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.328645    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:58:50.598925 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:50.598821    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"]
Sep 04 23:58:50.601917 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:58:50.601874    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92ef599b_cb26_42aa_b008_2f317bdf2b59.slice/crio-ede3e78ea85c390f5ed9a804f8914f106d603b04941933b48138379d22968c29 WatchSource:0}: Error finding container ede3e78ea85c390f5ed9a804f8914f106d603b04941933b48138379d22968c29: Status 404 returned error can't find the container with id ede3e78ea85c390f5ed9a804f8914f106d603b04941933b48138379d22968c29
Sep 04 23:58:51.581404 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:51.581370    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl" event={"ID":"92ef599b-cb26-42aa-b008-2f317bdf2b59","Type":"ContainerStarted","Data":"7d37a120ea0cd8b541360c3e5bfdbd1f5112b77f451f4b48bb7ec51f4cbe05e3"}
Sep 04 23:58:51.581821 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:51.581411    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl" event={"ID":"92ef599b-cb26-42aa-b008-2f317bdf2b59","Type":"ContainerStarted","Data":"ede3e78ea85c390f5ed9a804f8914f106d603b04941933b48138379d22968c29"}
Sep 04 23:58:51.596222 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:51.596092    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl" podStartSLOduration=2.596076322 podStartE2EDuration="2.596076322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:58:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:58:51.595488323 +0000 UTC m=+1319.843488860" watchObservedRunningTime="2026-09-04 23:58:51.596076322 +0000 UTC m=+1319.844076838"
Sep 04 23:58:54.138863 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:54.138830    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:55.045312 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.045237    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6" podUID="946c686c-9a43-4442-9d88-c24f66d0bf2a" containerName="main" containerID="cri-o://879f5672ef35e29504b0f3326bb5e6a1c1755937cb3dbacb29ade44050704854" gracePeriod=45
Sep 04 23:58:55.328448 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.328427    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:58:55.340079 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.340057    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-home\") pod \"946c686c-9a43-4442-9d88-c24f66d0bf2a\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") "
Sep 04 23:58:55.340141 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.340104    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-tmp-dir\") pod \"946c686c-9a43-4442-9d88-c24f66d0bf2a\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") "
Sep 04 23:58:55.340141 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.340125    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-model-cache\") pod \"946c686c-9a43-4442-9d88-c24f66d0bf2a\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") "
Sep 04 23:58:55.340217 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.340194    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlwn7\" (UniqueName: \"kubernetes.io/projected/946c686c-9a43-4442-9d88-c24f66d0bf2a-kube-api-access-jlwn7\") pod \"946c686c-9a43-4442-9d88-c24f66d0bf2a\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") "
Sep 04 23:58:55.340217 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.340212    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/946c686c-9a43-4442-9d88-c24f66d0bf2a-tls-certs\") pod \"946c686c-9a43-4442-9d88-c24f66d0bf2a\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") "
Sep 04 23:58:55.340327 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.340242    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-dshm\") pod \"946c686c-9a43-4442-9d88-c24f66d0bf2a\" (UID: \"946c686c-9a43-4442-9d88-c24f66d0bf2a\") "
Sep 04 23:58:55.340327 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.340298    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "946c686c-9a43-4442-9d88-c24f66d0bf2a" (UID: "946c686c-9a43-4442-9d88-c24f66d0bf2a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:58:55.340431 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.340312    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-home" (OuterVolumeSpecName: "home") pod "946c686c-9a43-4442-9d88-c24f66d0bf2a" (UID: "946c686c-9a43-4442-9d88-c24f66d0bf2a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:58:55.340431 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.340387    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-model-cache" (OuterVolumeSpecName: "model-cache") pod "946c686c-9a43-4442-9d88-c24f66d0bf2a" (UID: "946c686c-9a43-4442-9d88-c24f66d0bf2a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:58:55.340554 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.340534    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-home\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:58:55.340596 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.340559    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-tmp-dir\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:58:55.340596 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.340573    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-model-cache\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:58:55.342457 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.342429    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-dshm" (OuterVolumeSpecName: "dshm") pod "946c686c-9a43-4442-9d88-c24f66d0bf2a" (UID: "946c686c-9a43-4442-9d88-c24f66d0bf2a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 23:58:55.342566 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.342521    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/946c686c-9a43-4442-9d88-c24f66d0bf2a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "946c686c-9a43-4442-9d88-c24f66d0bf2a" (UID: "946c686c-9a43-4442-9d88-c24f66d0bf2a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 23:58:55.342710 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.342686    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/946c686c-9a43-4442-9d88-c24f66d0bf2a-kube-api-access-jlwn7" (OuterVolumeSpecName: "kube-api-access-jlwn7") pod "946c686c-9a43-4442-9d88-c24f66d0bf2a" (UID: "946c686c-9a43-4442-9d88-c24f66d0bf2a"). InnerVolumeSpecName "kube-api-access-jlwn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:58:55.441870 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.441838    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlwn7\" (UniqueName: \"kubernetes.io/projected/946c686c-9a43-4442-9d88-c24f66d0bf2a-kube-api-access-jlwn7\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:58:55.441870 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.441866    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/946c686c-9a43-4442-9d88-c24f66d0bf2a-tls-certs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:58:55.441870 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.441875    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/946c686c-9a43-4442-9d88-c24f66d0bf2a-dshm\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:58:55.603422 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.603345    2564 generic.go:356] "Generic (PLEG): container finished" podID="946c686c-9a43-4442-9d88-c24f66d0bf2a" containerID="879f5672ef35e29504b0f3326bb5e6a1c1755937cb3dbacb29ade44050704854" exitCode=0
Sep 04 23:58:55.603422 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.603395    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"
Sep 04 23:58:55.603599 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.603426    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6" event={"ID":"946c686c-9a43-4442-9d88-c24f66d0bf2a","Type":"ContainerDied","Data":"879f5672ef35e29504b0f3326bb5e6a1c1755937cb3dbacb29ade44050704854"}
Sep 04 23:58:55.603599 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.603464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6" event={"ID":"946c686c-9a43-4442-9d88-c24f66d0bf2a","Type":"ContainerDied","Data":"a8af51928a957748d754cb88962a82b613578aec6516dca3f270157a45aaaf75"}
Sep 04 23:58:55.603599 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.603486    2564 scope.go:117] "RemoveContainer" containerID="879f5672ef35e29504b0f3326bb5e6a1c1755937cb3dbacb29ade44050704854"
Sep 04 23:58:55.614253 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.614229    2564 scope.go:117] "RemoveContainer" containerID="879f5672ef35e29504b0f3326bb5e6a1c1755937cb3dbacb29ade44050704854"
Sep 04 23:58:55.614542 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:58:55.614522    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"879f5672ef35e29504b0f3326bb5e6a1c1755937cb3dbacb29ade44050704854\": container with ID starting with 879f5672ef35e29504b0f3326bb5e6a1c1755937cb3dbacb29ade44050704854 not found: ID does not exist" containerID="879f5672ef35e29504b0f3326bb5e6a1c1755937cb3dbacb29ade44050704854"
Sep 04 23:58:55.614591 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.614552    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"879f5672ef35e29504b0f3326bb5e6a1c1755937cb3dbacb29ade44050704854"} err="failed to get container status \"879f5672ef35e29504b0f3326bb5e6a1c1755937cb3dbacb29ade44050704854\": rpc error: code = NotFound desc = could not find container \"879f5672ef35e29504b0f3326bb5e6a1c1755937cb3dbacb29ade44050704854\": container with ID starting with 879f5672ef35e29504b0f3326bb5e6a1c1755937cb3dbacb29ade44050704854 not found: ID does not exist"
Sep 04 23:58:55.625831 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.625811    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"]
Sep 04 23:58:55.630872 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:55.630850    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5245c1/sub-test-model-5245c1-kserve-76dc98694c-cbng6"]
Sep 04 23:58:56.170965 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:56.170924    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:56.175336 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:56.175309    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="946c686c-9a43-4442-9d88-c24f66d0bf2a" path="/var/lib/kubelet/pods/946c686c-9a43-4442-9d88-c24f66d0bf2a/volumes"
Sep 04 23:58:59.180063 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.180032    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn"]
Sep 04 23:58:59.180437 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.180424    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="946c686c-9a43-4442-9d88-c24f66d0bf2a" containerName="main"
Sep 04 23:58:59.180482 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.180438    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="946c686c-9a43-4442-9d88-c24f66d0bf2a" containerName="main"
Sep 04 23:58:59.180520 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.180502    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="946c686c-9a43-4442-9d88-c24f66d0bf2a" containerName="main"
Sep 04 23:58:59.184796 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.184776    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn"
Sep 04 23:58:59.188065 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.188045    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k4f64\""
Sep 04 23:58:59.188231 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.188110    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 23:58:59.189421 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.189394    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn"]
Sep 04 23:58:59.273568 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.273541    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a244d62e-f8ef-44c8-8f68-d54f24fbf7ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn\" (UID: \"a244d62e-f8ef-44c8-8f68-d54f24fbf7ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn"
Sep 04 23:58:59.373914 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.373878    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a244d62e-f8ef-44c8-8f68-d54f24fbf7ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn\" (UID: \"a244d62e-f8ef-44c8-8f68-d54f24fbf7ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn"
Sep 04 23:58:59.374474 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.374454    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a244d62e-f8ef-44c8-8f68-d54f24fbf7ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn\" (UID: \"a244d62e-f8ef-44c8-8f68-d54f24fbf7ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn"
Sep 04 23:58:59.496736 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.496684    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn"
Sep 04 23:58:59.509193 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.509170    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:58:59.767381 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:58:59.767350    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn"]
Sep 04 23:58:59.769258 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:58:59.769218    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda244d62e_f8ef_44c8_8f68_d54f24fbf7ac.slice/crio-bb8f96066f55695926ff36ff8fd77a23b9afa3922f692fbbf5814c06866463da WatchSource:0}: Error finding container bb8f96066f55695926ff36ff8fd77a23b9afa3922f692fbbf5814c06866463da: Status 404 returned error can't find the container with id bb8f96066f55695926ff36ff8fd77a23b9afa3922f692fbbf5814c06866463da
Sep 04 23:59:00.329010 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:00.328936    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:59:00.329010 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:00.328995    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:59:00.341377 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:00.341356    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:59:00.636182 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:00.636145    2564 generic.go:356] "Generic (PLEG): container finished" podID="a244d62e-f8ef-44c8-8f68-d54f24fbf7ac" containerID="2159960ded29dd0b23153320dfc4b47728f15d2c9d86183af04311edc57b0445" exitCode=0
Sep 04 23:59:00.636329 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:00.636221    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn" event={"ID":"a244d62e-f8ef-44c8-8f68-d54f24fbf7ac","Type":"ContainerDied","Data":"2159960ded29dd0b23153320dfc4b47728f15d2c9d86183af04311edc57b0445"}
Sep 04 23:59:00.636329 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:00.636269    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn" event={"ID":"a244d62e-f8ef-44c8-8f68-d54f24fbf7ac","Type":"ContainerStarted","Data":"bb8f96066f55695926ff36ff8fd77a23b9afa3922f692fbbf5814c06866463da"}
Sep 04 23:59:00.647990 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:00.647970    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 04 23:59:01.784847 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:01.784824    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn"
Sep 04 23:59:01.891767 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:01.891737    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a244d62e-f8ef-44c8-8f68-d54f24fbf7ac-maas-api-service-ca\") pod \"a244d62e-f8ef-44c8-8f68-d54f24fbf7ac\" (UID: \"a244d62e-f8ef-44c8-8f68-d54f24fbf7ac\") "
Sep 04 23:59:01.892069 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:01.892050    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a244d62e-f8ef-44c8-8f68-d54f24fbf7ac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a244d62e-f8ef-44c8-8f68-d54f24fbf7ac" (UID: "a244d62e-f8ef-44c8-8f68-d54f24fbf7ac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:59:01.992647 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:01.992592    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a244d62e-f8ef-44c8-8f68-d54f24fbf7ac-maas-api-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:59:02.298569 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:02.298483    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:02.648014 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:02.647986    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn"
Sep 04 23:59:02.648177 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:02.648025    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn" event={"ID":"a244d62e-f8ef-44c8-8f68-d54f24fbf7ac","Type":"ContainerDied","Data":"bb8f96066f55695926ff36ff8fd77a23b9afa3922f692fbbf5814c06866463da"}
Sep 04 23:59:02.648177 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:02.648073    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb8f96066f55695926ff36ff8fd77a23b9afa3922f692fbbf5814c06866463da"
Sep 04 23:59:09.309337 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:09.309304    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:10.253264 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:10.253228    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:10.529556 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:10.529474    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:14.848047 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:14.847967    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:14.878390 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:14.878361    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"]
Sep 04 23:59:14.878597 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:14.878557    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j" podUID="677b6701-ac1e-4d07-b385-9f7953e3cb92" containerName="payload-processing" containerID="cri-o://6fe44b1fd9e672bc8cd9f166895dff7fdcdaff30308950108848eedeab3d7ed1" gracePeriod=30
Sep 04 23:59:14.883064 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:14.882514    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"]
Sep 04 23:59:14.883064 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:14.882849    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8" podUID="b3b33412-37db-4ba0-a455-0fbedd26925d" containerName="payload-pre-processing" containerID="cri-o://2c25601dd9ffea048bb3181de03af1f25efce96285c5d7bf38f0785c7003fdb6" gracePeriod=30
Sep 04 23:59:15.023791 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.023754    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn"]
Sep 04 23:59:15.025730 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.025707    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-87b0d1-c6mpn"]
Sep 04 23:59:15.166457 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.166434    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"
Sep 04 23:59:15.169348 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.169320    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"
Sep 04 23:59:15.302719 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.302680    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8kdp\" (UniqueName: \"kubernetes.io/projected/677b6701-ac1e-4d07-b385-9f7953e3cb92-kube-api-access-c8kdp\") pod \"677b6701-ac1e-4d07-b385-9f7953e3cb92\" (UID: \"677b6701-ac1e-4d07-b385-9f7953e3cb92\") "
Sep 04 23:59:15.302934 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.302748    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/677b6701-ac1e-4d07-b385-9f7953e3cb92-plugins-config-volume\") pod \"677b6701-ac1e-4d07-b385-9f7953e3cb92\" (UID: \"677b6701-ac1e-4d07-b385-9f7953e3cb92\") "
Sep 04 23:59:15.302934 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.302777    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62vh6\" (UniqueName: \"kubernetes.io/projected/b3b33412-37db-4ba0-a455-0fbedd26925d-kube-api-access-62vh6\") pod \"b3b33412-37db-4ba0-a455-0fbedd26925d\" (UID: \"b3b33412-37db-4ba0-a455-0fbedd26925d\") "
Sep 04 23:59:15.302934 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.302811    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3b33412-37db-4ba0-a455-0fbedd26925d-plugins-config-volume\") pod \"b3b33412-37db-4ba0-a455-0fbedd26925d\" (UID: \"b3b33412-37db-4ba0-a455-0fbedd26925d\") "
Sep 04 23:59:15.303206 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.303174    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/677b6701-ac1e-4d07-b385-9f7953e3cb92-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "677b6701-ac1e-4d07-b385-9f7953e3cb92" (UID: "677b6701-ac1e-4d07-b385-9f7953e3cb92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:59:15.303285 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.303216    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3b33412-37db-4ba0-a455-0fbedd26925d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b3b33412-37db-4ba0-a455-0fbedd26925d" (UID: "b3b33412-37db-4ba0-a455-0fbedd26925d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:59:15.305111 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.305084    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3b33412-37db-4ba0-a455-0fbedd26925d-kube-api-access-62vh6" (OuterVolumeSpecName: "kube-api-access-62vh6") pod "b3b33412-37db-4ba0-a455-0fbedd26925d" (UID: "b3b33412-37db-4ba0-a455-0fbedd26925d"). InnerVolumeSpecName "kube-api-access-62vh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:59:15.305188 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.305101    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/677b6701-ac1e-4d07-b385-9f7953e3cb92-kube-api-access-c8kdp" (OuterVolumeSpecName: "kube-api-access-c8kdp") pod "677b6701-ac1e-4d07-b385-9f7953e3cb92" (UID: "677b6701-ac1e-4d07-b385-9f7953e3cb92"). InnerVolumeSpecName "kube-api-access-c8kdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:59:15.308772 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.308738    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:15.403965 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.403914    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62vh6\" (UniqueName: \"kubernetes.io/projected/b3b33412-37db-4ba0-a455-0fbedd26925d-kube-api-access-62vh6\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:59:15.403965 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.403937    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3b33412-37db-4ba0-a455-0fbedd26925d-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:59:15.403965 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.403947    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8kdp\" (UniqueName: \"kubernetes.io/projected/677b6701-ac1e-4d07-b385-9f7953e3cb92-kube-api-access-c8kdp\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:59:15.403965 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.403964    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/677b6701-ac1e-4d07-b385-9f7953e3cb92-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:59:15.409278 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.409254    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:15.718054 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.717990    2564 generic.go:356] "Generic (PLEG): container finished" podID="677b6701-ac1e-4d07-b385-9f7953e3cb92" containerID="6fe44b1fd9e672bc8cd9f166895dff7fdcdaff30308950108848eedeab3d7ed1" exitCode=0
Sep 04 23:59:15.718054 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.718030    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"
Sep 04 23:59:15.718212 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.718077    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j" event={"ID":"677b6701-ac1e-4d07-b385-9f7953e3cb92","Type":"ContainerDied","Data":"6fe44b1fd9e672bc8cd9f166895dff7fdcdaff30308950108848eedeab3d7ed1"}
Sep 04 23:59:15.718212 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.718139    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j" event={"ID":"677b6701-ac1e-4d07-b385-9f7953e3cb92","Type":"ContainerDied","Data":"dd2594e050ecc916d130e0c8006b959f955d6b02eb2be7e1ea6ec958be2cbf4b"}
Sep 04 23:59:15.718212 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.718166    2564 scope.go:117] "RemoveContainer" containerID="6fe44b1fd9e672bc8cd9f166895dff7fdcdaff30308950108848eedeab3d7ed1"
Sep 04 23:59:15.719430 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.719350    2564 generic.go:356] "Generic (PLEG): container finished" podID="b3b33412-37db-4ba0-a455-0fbedd26925d" containerID="2c25601dd9ffea048bb3181de03af1f25efce96285c5d7bf38f0785c7003fdb6" exitCode=0
Sep 04 23:59:15.719430 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.719385    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8" event={"ID":"b3b33412-37db-4ba0-a455-0fbedd26925d","Type":"ContainerDied","Data":"2c25601dd9ffea048bb3181de03af1f25efce96285c5d7bf38f0785c7003fdb6"}
Sep 04 23:59:15.719528 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.719446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8" event={"ID":"b3b33412-37db-4ba0-a455-0fbedd26925d","Type":"ContainerDied","Data":"358ac5804d4473e015f87c07a87806042982133a4409f461747f04f2435d620a"}
Sep 04 23:59:15.719528 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.719395    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"
Sep 04 23:59:15.727099 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.727080    2564 scope.go:117] "RemoveContainer" containerID="6fe44b1fd9e672bc8cd9f166895dff7fdcdaff30308950108848eedeab3d7ed1"
Sep 04 23:59:15.727372 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:59:15.727354    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fe44b1fd9e672bc8cd9f166895dff7fdcdaff30308950108848eedeab3d7ed1\": container with ID starting with 6fe44b1fd9e672bc8cd9f166895dff7fdcdaff30308950108848eedeab3d7ed1 not found: ID does not exist" containerID="6fe44b1fd9e672bc8cd9f166895dff7fdcdaff30308950108848eedeab3d7ed1"
Sep 04 23:59:15.727440 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.727380    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe44b1fd9e672bc8cd9f166895dff7fdcdaff30308950108848eedeab3d7ed1"} err="failed to get container status \"6fe44b1fd9e672bc8cd9f166895dff7fdcdaff30308950108848eedeab3d7ed1\": rpc error: code = NotFound desc = could not find container \"6fe44b1fd9e672bc8cd9f166895dff7fdcdaff30308950108848eedeab3d7ed1\": container with ID starting with 6fe44b1fd9e672bc8cd9f166895dff7fdcdaff30308950108848eedeab3d7ed1 not found: ID does not exist"
Sep 04 23:59:15.727440 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.727396    2564 scope.go:117] "RemoveContainer" containerID="2c25601dd9ffea048bb3181de03af1f25efce96285c5d7bf38f0785c7003fdb6"
Sep 04 23:59:15.735249 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.735231    2564 scope.go:117] "RemoveContainer" containerID="2c25601dd9ffea048bb3181de03af1f25efce96285c5d7bf38f0785c7003fdb6"
Sep 04 23:59:15.735536 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:59:15.735520    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c25601dd9ffea048bb3181de03af1f25efce96285c5d7bf38f0785c7003fdb6\": container with ID starting with 2c25601dd9ffea048bb3181de03af1f25efce96285c5d7bf38f0785c7003fdb6 not found: ID does not exist" containerID="2c25601dd9ffea048bb3181de03af1f25efce96285c5d7bf38f0785c7003fdb6"
Sep 04 23:59:15.735592 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.735544    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c25601dd9ffea048bb3181de03af1f25efce96285c5d7bf38f0785c7003fdb6"} err="failed to get container status \"2c25601dd9ffea048bb3181de03af1f25efce96285c5d7bf38f0785c7003fdb6\": rpc error: code = NotFound desc = could not find container \"2c25601dd9ffea048bb3181de03af1f25efce96285c5d7bf38f0785c7003fdb6\": container with ID starting with 2c25601dd9ffea048bb3181de03af1f25efce96285c5d7bf38f0785c7003fdb6 not found: ID does not exist"
Sep 04 23:59:15.741501 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.741479    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"]
Sep 04 23:59:15.747338 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.747316    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-87b0d1-7c47989878-bkhx8"]
Sep 04 23:59:15.757133 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.757113    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"]
Sep 04 23:59:15.760610 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:15.760591    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-87b0d1-6bf8ff5575-tl85j"]
Sep 04 23:59:16.177026 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:16.176993    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="677b6701-ac1e-4d07-b385-9f7953e3cb92" path="/var/lib/kubelet/pods/677b6701-ac1e-4d07-b385-9f7953e3cb92/volumes"
Sep 04 23:59:16.177436 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:16.177419    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a244d62e-f8ef-44c8-8f68-d54f24fbf7ac" path="/var/lib/kubelet/pods/a244d62e-f8ef-44c8-8f68-d54f24fbf7ac/volumes"
Sep 04 23:59:16.177868 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:16.177852    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3b33412-37db-4ba0-a455-0fbedd26925d" path="/var/lib/kubelet/pods/b3b33412-37db-4ba0-a455-0fbedd26925d/volumes"
Sep 04 23:59:20.988330 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:20.988296    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:21.088335 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.088301    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:21.229887 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.229854    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:21.559993 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.559963    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs"]
Sep 04 23:59:21.560341 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.560326    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="677b6701-ac1e-4d07-b385-9f7953e3cb92" containerName="payload-processing"
Sep 04 23:59:21.560341 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.560339    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="677b6701-ac1e-4d07-b385-9f7953e3cb92" containerName="payload-processing"
Sep 04 23:59:21.560493 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.560355    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a244d62e-f8ef-44c8-8f68-d54f24fbf7ac" containerName="revoke-keys"
Sep 04 23:59:21.560493 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.560361    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a244d62e-f8ef-44c8-8f68-d54f24fbf7ac" containerName="revoke-keys"
Sep 04 23:59:21.560493 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.560368    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3b33412-37db-4ba0-a455-0fbedd26925d" containerName="payload-pre-processing"
Sep 04 23:59:21.560493 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.560374    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3b33412-37db-4ba0-a455-0fbedd26925d" containerName="payload-pre-processing"
Sep 04 23:59:21.560493 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.560437    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="677b6701-ac1e-4d07-b385-9f7953e3cb92" containerName="payload-processing"
Sep 04 23:59:21.560493 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.560446    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a244d62e-f8ef-44c8-8f68-d54f24fbf7ac" containerName="revoke-keys"
Sep 04 23:59:21.560493 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.560452    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3b33412-37db-4ba0-a455-0fbedd26925d" containerName="payload-pre-processing"
Sep 04 23:59:21.563672 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.563645    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs"
Sep 04 23:59:21.566120 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.566098    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 23:59:21.566245 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.566153    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k4f64\""
Sep 04 23:59:21.570417 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.570390    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs"]
Sep 04 23:59:21.659754 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.659727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0f211a-16a4-48ae-a072-eaf71abe74f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs\" (UID: \"bf0f211a-16a4-48ae-a072-eaf71abe74f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs"
Sep 04 23:59:21.689348 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.689314    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:21.761005 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.760972    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0f211a-16a4-48ae-a072-eaf71abe74f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs\" (UID: \"bf0f211a-16a4-48ae-a072-eaf71abe74f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs"
Sep 04 23:59:21.761768 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.761738    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0f211a-16a4-48ae-a072-eaf71abe74f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs\" (UID: \"bf0f211a-16a4-48ae-a072-eaf71abe74f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs"
Sep 04 23:59:21.876449 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:21.876426    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs"
Sep 04 23:59:22.144774 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:22.144710    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs"]
Sep 04 23:59:22.147460 ip-10-0-139-127 kubenswrapper[2564]: W0904 23:59:22.147439    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0f211a_16a4_48ae_a072_eaf71abe74f2.slice/crio-8d0fc52d23c53b6c3b2efadfc13312f70a17e9918b6f61d95a734b9febf128a0 WatchSource:0}: Error finding container 8d0fc52d23c53b6c3b2efadfc13312f70a17e9918b6f61d95a734b9febf128a0: Status 404 returned error can't find the container with id 8d0fc52d23c53b6c3b2efadfc13312f70a17e9918b6f61d95a734b9febf128a0
Sep 04 23:59:22.761724 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:22.761639    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs" event={"ID":"bf0f211a-16a4-48ae-a072-eaf71abe74f2","Type":"ContainerStarted","Data":"fbfa042e9a425c43f5863e809b51c695163f574af5f1da91efa3b48299b4b90b"}
Sep 04 23:59:22.761889 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:22.761743    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs" event={"ID":"bf0f211a-16a4-48ae-a072-eaf71abe74f2","Type":"ContainerStarted","Data":"8d0fc52d23c53b6c3b2efadfc13312f70a17e9918b6f61d95a734b9febf128a0"}
Sep 04 23:59:22.777806 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:22.777760    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs" podStartSLOduration=1.777745426 podStartE2EDuration="1.777745426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 23:59:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 23:59:22.776238789 +0000 UTC m=+1351.024239306" watchObservedRunningTime="2026-09-04 23:59:22.777745426 +0000 UTC m=+1351.025745947"
Sep 04 23:59:23.769399 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:23.769364    2564 generic.go:356] "Generic (PLEG): container finished" podID="bf0f211a-16a4-48ae-a072-eaf71abe74f2" containerID="fbfa042e9a425c43f5863e809b51c695163f574af5f1da91efa3b48299b4b90b" exitCode=0
Sep 04 23:59:23.769735 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:23.769455    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs" event={"ID":"bf0f211a-16a4-48ae-a072-eaf71abe74f2","Type":"ContainerDied","Data":"fbfa042e9a425c43f5863e809b51c695163f574af5f1da91efa3b48299b4b90b"}
Sep 04 23:59:24.906557 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:24.906535    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs"
Sep 04 23:59:24.986296 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:24.986264    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0f211a-16a4-48ae-a072-eaf71abe74f2-maas-api-service-ca\") pod \"bf0f211a-16a4-48ae-a072-eaf71abe74f2\" (UID: \"bf0f211a-16a4-48ae-a072-eaf71abe74f2\") "
Sep 04 23:59:24.986590 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:24.986568    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf0f211a-16a4-48ae-a072-eaf71abe74f2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bf0f211a-16a4-48ae-a072-eaf71abe74f2" (UID: "bf0f211a-16a4-48ae-a072-eaf71abe74f2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:59:25.087534 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:25.087475    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0f211a-16a4-48ae-a072-eaf71abe74f2-maas-api-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:59:25.780615 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:25.780588    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs"
Sep 04 23:59:25.780615 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:25.780589    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs" event={"ID":"bf0f211a-16a4-48ae-a072-eaf71abe74f2","Type":"ContainerDied","Data":"8d0fc52d23c53b6c3b2efadfc13312f70a17e9918b6f61d95a734b9febf128a0"}
Sep 04 23:59:25.780830 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:25.780632    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d0fc52d23c53b6c3b2efadfc13312f70a17e9918b6f61d95a734b9febf128a0"
Sep 04 23:59:25.822663 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:59:25.822635    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0f211a_16a4_48ae_a072_eaf71abe74f2.slice\": RecentStats: unable to find data in memory cache]"
Sep 04 23:59:30.342091 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:30.342061    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:31.697639 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:31.697602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:37.242866 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:37.242832    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:37.339670 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:37.339637    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:37.407533 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:37.407497    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs"]
Sep 04 23:59:37.410050 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:37.410017    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:37.410266 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:37.410247    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8cff3b-qdtqs"]
Sep 04 23:59:37.470056 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:37.470018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:37.777474 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:37.777413    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:38.175568 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:38.175535    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf0f211a-16a4-48ae-a072-eaf71abe74f2" path="/var/lib/kubelet/pods/bf0f211a-16a4-48ae-a072-eaf71abe74f2/volumes"
Sep 04 23:59:43.399140 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:43.399106    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:43.474111 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:43.474080    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:43.716873 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:43.716787    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:44.338762 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:44.338728    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:52.348886 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:52.348860    2564 scope.go:117] "RemoveContainer" containerID="5b45c3f63fe790d5c980c3f2c6d529e81870141e276b9baacb13415dcdc5202d"
Sep 04 23:59:52.357021 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:52.356999    2564 scope.go:117] "RemoveContainer" containerID="322ebedae192c3ee498bb21794d2df5127ee3ec0115fcb6884b1a0072b8e702d"
Sep 04 23:59:54.419330 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.419293    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:54.437334 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.437308    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"]
Sep 04 23:59:54.437524 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.437502    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp" podUID="80c32a2d-405b-4e21-b12d-7e94189a1ba8" containerName="payload-processing" containerID="cri-o://68669c96457e7b77a00e03adb02a02585faa11c5ec1e8516571ad2e23d68752e" gracePeriod=30
Sep 04 23:59:54.521135 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.521106    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:54.586193 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.586153    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:54.643421 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.643393    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:54.697939 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.697887    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"
Sep 04 23:59:54.737335 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.737314    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8gv5\" (UniqueName: \"kubernetes.io/projected/80c32a2d-405b-4e21-b12d-7e94189a1ba8-kube-api-access-p8gv5\") pod \"80c32a2d-405b-4e21-b12d-7e94189a1ba8\" (UID: \"80c32a2d-405b-4e21-b12d-7e94189a1ba8\") "
Sep 04 23:59:54.737452 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.737357    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c32a2d-405b-4e21-b12d-7e94189a1ba8-plugins-config-volume\") pod \"80c32a2d-405b-4e21-b12d-7e94189a1ba8\" (UID: \"80c32a2d-405b-4e21-b12d-7e94189a1ba8\") "
Sep 04 23:59:54.737726 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.737707    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80c32a2d-405b-4e21-b12d-7e94189a1ba8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "80c32a2d-405b-4e21-b12d-7e94189a1ba8" (UID: "80c32a2d-405b-4e21-b12d-7e94189a1ba8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 23:59:54.739495 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.739474    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80c32a2d-405b-4e21-b12d-7e94189a1ba8-kube-api-access-p8gv5" (OuterVolumeSpecName: "kube-api-access-p8gv5") pod "80c32a2d-405b-4e21-b12d-7e94189a1ba8" (UID: "80c32a2d-405b-4e21-b12d-7e94189a1ba8"). InnerVolumeSpecName "kube-api-access-p8gv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 23:59:54.838964 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.838938    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8gv5\" (UniqueName: \"kubernetes.io/projected/80c32a2d-405b-4e21-b12d-7e94189a1ba8-kube-api-access-p8gv5\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:59:54.838964 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.838962    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c32a2d-405b-4e21-b12d-7e94189a1ba8-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 04 23:59:54.915635 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.915602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 04 23:59:54.925570 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.925547    2564 generic.go:356] "Generic (PLEG): container finished" podID="80c32a2d-405b-4e21-b12d-7e94189a1ba8" containerID="68669c96457e7b77a00e03adb02a02585faa11c5ec1e8516571ad2e23d68752e" exitCode=0
Sep 04 23:59:54.925661 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.925602    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"
Sep 04 23:59:54.925661 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.925640    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp" event={"ID":"80c32a2d-405b-4e21-b12d-7e94189a1ba8","Type":"ContainerDied","Data":"68669c96457e7b77a00e03adb02a02585faa11c5ec1e8516571ad2e23d68752e"}
Sep 04 23:59:54.925739 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.925719    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp" event={"ID":"80c32a2d-405b-4e21-b12d-7e94189a1ba8","Type":"ContainerDied","Data":"8f47b1945c25b45f714d89d4a29490e050b6502511352480263de8f473d52d0c"}
Sep 04 23:59:54.925774 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.925746    2564 scope.go:117] "RemoveContainer" containerID="68669c96457e7b77a00e03adb02a02585faa11c5ec1e8516571ad2e23d68752e"
Sep 04 23:59:54.934371 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.934356    2564 scope.go:117] "RemoveContainer" containerID="68669c96457e7b77a00e03adb02a02585faa11c5ec1e8516571ad2e23d68752e"
Sep 04 23:59:54.934631 ip-10-0-139-127 kubenswrapper[2564]: E0904 23:59:54.934612    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68669c96457e7b77a00e03adb02a02585faa11c5ec1e8516571ad2e23d68752e\": container with ID starting with 68669c96457e7b77a00e03adb02a02585faa11c5ec1e8516571ad2e23d68752e not found: ID does not exist" containerID="68669c96457e7b77a00e03adb02a02585faa11c5ec1e8516571ad2e23d68752e"
Sep 04 23:59:54.934691 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.934638    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68669c96457e7b77a00e03adb02a02585faa11c5ec1e8516571ad2e23d68752e"} err="failed to get container status \"68669c96457e7b77a00e03adb02a02585faa11c5ec1e8516571ad2e23d68752e\": rpc error: code = NotFound desc = could not find container \"68669c96457e7b77a00e03adb02a02585faa11c5ec1e8516571ad2e23d68752e\": container with ID starting with 68669c96457e7b77a00e03adb02a02585faa11c5ec1e8516571ad2e23d68752e not found: ID does not exist"
Sep 04 23:59:54.945743 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.945723    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"]
Sep 04 23:59:54.949117 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:54.949097    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e35897-cbffdc47f-n2sqp"]
Sep 04 23:59:56.174961 ip-10-0-139-127 kubenswrapper[2564]: I0904 23:59:56.174926    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80c32a2d-405b-4e21-b12d-7e94189a1ba8" path="/var/lib/kubelet/pods/80c32a2d-405b-4e21-b12d-7e94189a1ba8/volumes"
Sep 05 00:00:00.137966 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.137936    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd"]
Sep 05 00:00:00.138493 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.138477    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf0f211a-16a4-48ae-a072-eaf71abe74f2" containerName="revoke-keys"
Sep 05 00:00:00.138540 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.138497    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf0f211a-16a4-48ae-a072-eaf71abe74f2" containerName="revoke-keys"
Sep 05 00:00:00.138540 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.138527    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80c32a2d-405b-4e21-b12d-7e94189a1ba8" containerName="payload-processing"
Sep 05 00:00:00.138540 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.138535    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="80c32a2d-405b-4e21-b12d-7e94189a1ba8" containerName="payload-processing"
Sep 05 00:00:00.138645 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.138633    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf0f211a-16a4-48ae-a072-eaf71abe74f2" containerName="revoke-keys"
Sep 05 00:00:00.138688 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.138652    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="80c32a2d-405b-4e21-b12d-7e94189a1ba8" containerName="payload-processing"
Sep 05 00:00:00.143515 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.143500    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809440-cxv58"]
Sep 05 00:00:00.143636 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.143621    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd"
Sep 05 00:00:00.146120 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.146097    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k4f64\""
Sep 05 00:00:00.147000 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.146982    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809440-cxv58"
Sep 05 00:00:00.149678 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.149653    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd"]
Sep 05 00:00:00.151952 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.151935    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809440-cxv58"]
Sep 05 00:00:00.183320 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.183297    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd"
Sep 05 00:00:00.189856 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.189837    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809440-cxv58"
Sep 05 00:00:00.463892 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.463845    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd"]
Sep 05 00:00:00.468280 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.468079    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809440-cxv58"]
Sep 05 00:00:00.652192 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.652158    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:00.803154 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.803117    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:00.957058 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.957026    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809440-cxv58" event={"ID":"80df0ae5-caec-4846-963e-053de587bb78","Type":"ContainerStarted","Data":"5c5a11be7f5068fe5620a29185c6ee8c8feab1d38156502018e73a1ea704ecf0"}
Sep 05 00:00:00.957211 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.957085    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809440-cxv58" event={"ID":"80df0ae5-caec-4846-963e-053de587bb78","Type":"ContainerStarted","Data":"731cae7ea73aa47fbb1c15ef22d8fa532428cd17caba9f00fafa313ca338c81d"}
Sep 05 00:00:00.958555 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.958529    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd" event={"ID":"f570bdc0-e8e4-4ecc-9cd9-fef771da8e9f","Type":"ContainerStarted","Data":"db6c6423ed4725f778fb761fbdb099d38066fa24e79398d4311eaa98dde7698d"}
Sep 05 00:00:00.958643 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.958570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd" event={"ID":"f570bdc0-e8e4-4ecc-9cd9-fef771da8e9f","Type":"ContainerStarted","Data":"f1103fab1b3e6522a595d34c420905657b76b6161e74671e8c08f8672a16d7fb"}
Sep 05 00:00:00.974360 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.974306    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809440-cxv58" podStartSLOduration=0.974287141 podStartE2EDuration="974.287141ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 00:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 00:00:00.971846317 +0000 UTC m=+1389.219846853" watchObservedRunningTime="2026-09-05 00:00:00.974287141 +0000 UTC m=+1389.222287658"
Sep 05 00:00:00.976262 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.976233    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:00.986120 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:00.986087    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd" podStartSLOduration=0.986075528 podStartE2EDuration="986.075528ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 00:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 00:00:00.985439252 +0000 UTC m=+1389.233439769" watchObservedRunningTime="2026-09-05 00:00:00.986075528 +0000 UTC m=+1389.234076044"
Sep 05 00:00:01.226217 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.226184    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2"]
Sep 05 00:00:01.229616 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.229598    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2"
Sep 05 00:00:01.232248 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.232225    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 00:00:01.236647 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.236624    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2"]
Sep 05 00:00:01.288858 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.288826    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2\" (UID: \"a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2"
Sep 05 00:00:01.390121 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.390090    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2\" (UID: \"a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2"
Sep 05 00:00:01.390738 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.390720    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2\" (UID: \"a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2"
Sep 05 00:00:01.403342 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.403315    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:01.542147 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.542081    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2"
Sep 05 00:00:01.819170 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.819138    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2"]
Sep 05 00:00:01.820882 ip-10-0-139-127 kubenswrapper[2564]: W0905 00:00:01.820850    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda13a2dd9_a5d0_482d_bbc6_bbc9feeaf7f0.slice/crio-6c68ec12606170e95a6a2d6893a81c772c73384d3bb0e33cdb19055ef21afc35 WatchSource:0}: Error finding container 6c68ec12606170e95a6a2d6893a81c772c73384d3bb0e33cdb19055ef21afc35: Status 404 returned error can't find the container with id 6c68ec12606170e95a6a2d6893a81c772c73384d3bb0e33cdb19055ef21afc35
Sep 05 00:00:01.965875 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.965829    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2" event={"ID":"a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0","Type":"ContainerStarted","Data":"6c68ec12606170e95a6a2d6893a81c772c73384d3bb0e33cdb19055ef21afc35"}
Sep 05 00:00:01.967665 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.967640    2564 generic.go:356] "Generic (PLEG): container finished" podID="f570bdc0-e8e4-4ecc-9cd9-fef771da8e9f" containerID="db6c6423ed4725f778fb761fbdb099d38066fa24e79398d4311eaa98dde7698d" exitCode=0
Sep 05 00:00:01.967798 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.967726    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd" event={"ID":"f570bdc0-e8e4-4ecc-9cd9-fef771da8e9f","Type":"ContainerDied","Data":"db6c6423ed4725f778fb761fbdb099d38066fa24e79398d4311eaa98dde7698d"}
Sep 05 00:00:01.969449 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.969427    2564 generic.go:356] "Generic (PLEG): container finished" podID="80df0ae5-caec-4846-963e-053de587bb78" containerID="5c5a11be7f5068fe5620a29185c6ee8c8feab1d38156502018e73a1ea704ecf0" exitCode=0
Sep 05 00:00:01.969531 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:01.969503    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809440-cxv58" event={"ID":"80df0ae5-caec-4846-963e-053de587bb78","Type":"ContainerDied","Data":"5c5a11be7f5068fe5620a29185c6ee8c8feab1d38156502018e73a1ea704ecf0"}
Sep 05 00:00:02.976158 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:02.976124    2564 generic.go:356] "Generic (PLEG): container finished" podID="a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0" containerID="9adfab0edc8fe0a76d062fbeafacda0565c6b6538c5f559449c3ee092e29710c" exitCode=0
Sep 05 00:00:02.976580 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:02.976218    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2" event={"ID":"a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0","Type":"ContainerDied","Data":"9adfab0edc8fe0a76d062fbeafacda0565c6b6538c5f559449c3ee092e29710c"}
Sep 05 00:00:03.114197 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:03.114173    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd"
Sep 05 00:00:03.138362 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:03.138341    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809440-cxv58"
Sep 05 00:00:03.510629 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:03.510595    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"]
Sep 05 00:00:03.658203 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:03.658172    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:03.982324 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:03.982286    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809440-cxv58"
Sep 05 00:00:03.982324 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:03.982310    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809440-cxv58" event={"ID":"80df0ae5-caec-4846-963e-053de587bb78","Type":"ContainerDied","Data":"731cae7ea73aa47fbb1c15ef22d8fa532428cd17caba9f00fafa313ca338c81d"}
Sep 05 00:00:03.982796 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:03.982361    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="731cae7ea73aa47fbb1c15ef22d8fa532428cd17caba9f00fafa313ca338c81d"
Sep 05 00:00:03.983950 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:03.983922    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd" event={"ID":"f570bdc0-e8e4-4ecc-9cd9-fef771da8e9f","Type":"ContainerDied","Data":"f1103fab1b3e6522a595d34c420905657b76b6161e74671e8c08f8672a16d7fb"}
Sep 05 00:00:03.983950 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:03.983944    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd"
Sep 05 00:00:03.984161 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:03.983971    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1103fab1b3e6522a595d34c420905657b76b6161e74671e8c08f8672a16d7fb"
Sep 05 00:00:04.121408 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:04.121387    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2"
Sep 05 00:00:04.215618 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:04.215595    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0-maas-api-service-ca\") pod \"a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0\" (UID: \"a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0\") "
Sep 05 00:00:04.215932 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:04.215890    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0" (UID: "a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 00:00:04.316859 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:04.316810    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0-maas-api-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:00:04.544874 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:04.544812    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:04.961163 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:04.961121    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:04.990484 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:04.990454    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2" event={"ID":"a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0","Type":"ContainerDied","Data":"6c68ec12606170e95a6a2d6893a81c772c73384d3bb0e33cdb19055ef21afc35"}
Sep 05 00:00:04.990799 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:04.990503    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c68ec12606170e95a6a2d6893a81c772c73384d3bb0e33cdb19055ef21afc35"
Sep 05 00:00:04.990799 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:04.990466    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2"
Sep 05 00:00:05.703714 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:05.703682    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:06.165144 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:06.165099    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:08.777415 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:08.777371    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:09.282946 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:09.282868    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:09.916811 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:09.916778    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:10.019309 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:10.019267    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:10.659573 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:10.659534    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:10.758493 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:10.758464    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:11.488074 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:11.488042    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:11.518831 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:11.518797    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"]
Sep 05 00:00:11.522323 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:11.522298    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"]
Sep 05 00:00:11.588192 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:11.588153    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:11.657919 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:11.657883    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2"]
Sep 05 00:00:11.666850 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:11.666820    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-f7eb8e-67wd2"]
Sep 05 00:00:12.024061 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.024026    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf" podUID="6bead5ba-e3c5-419e-b172-2d35682a9b48" containerName="payload-pre-processing" containerID="cri-o://b15de6052048c096f124737af856bd45f9caf9993a8b214881f95524c7c0a8ee" gracePeriod=30
Sep 05 00:00:12.024248 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.024079    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z" podUID="33e5df03-ba9b-4fef-a44e-d574a98141e1" containerName="payload-processing" containerID="cri-o://f544892ea5c1ebccacdabc2abb7582387974abe5dbf2982a09d4abe13c75cbc8" gracePeriod=30
Sep 05 00:00:12.177412 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.177383    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0" path="/var/lib/kubelet/pods/a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0/volumes"
Sep 05 00:00:12.256596 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.256559    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:12.308068 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.308044    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"
Sep 05 00:00:12.310132 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.310116    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"
Sep 05 00:00:12.326596 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.326568    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:12.388069 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.388035    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfrh5\" (UniqueName: \"kubernetes.io/projected/6bead5ba-e3c5-419e-b172-2d35682a9b48-kube-api-access-gfrh5\") pod \"6bead5ba-e3c5-419e-b172-2d35682a9b48\" (UID: \"6bead5ba-e3c5-419e-b172-2d35682a9b48\") "
Sep 05 00:00:12.388196 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.388088    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33e5df03-ba9b-4fef-a44e-d574a98141e1-plugins-config-volume\") pod \"33e5df03-ba9b-4fef-a44e-d574a98141e1\" (UID: \"33e5df03-ba9b-4fef-a44e-d574a98141e1\") "
Sep 05 00:00:12.388196 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.388177    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fflzj\" (UniqueName: \"kubernetes.io/projected/33e5df03-ba9b-4fef-a44e-d574a98141e1-kube-api-access-fflzj\") pod \"33e5df03-ba9b-4fef-a44e-d574a98141e1\" (UID: \"33e5df03-ba9b-4fef-a44e-d574a98141e1\") "
Sep 05 00:00:12.388196 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.388193    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bead5ba-e3c5-419e-b172-2d35682a9b48-plugins-config-volume\") pod \"6bead5ba-e3c5-419e-b172-2d35682a9b48\" (UID: \"6bead5ba-e3c5-419e-b172-2d35682a9b48\") "
Sep 05 00:00:12.388541 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.388510    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33e5df03-ba9b-4fef-a44e-d574a98141e1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "33e5df03-ba9b-4fef-a44e-d574a98141e1" (UID: "33e5df03-ba9b-4fef-a44e-d574a98141e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 00:00:12.388621 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.388560    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bead5ba-e3c5-419e-b172-2d35682a9b48-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6bead5ba-e3c5-419e-b172-2d35682a9b48" (UID: "6bead5ba-e3c5-419e-b172-2d35682a9b48"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 00:00:12.390258 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.390235    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33e5df03-ba9b-4fef-a44e-d574a98141e1-kube-api-access-fflzj" (OuterVolumeSpecName: "kube-api-access-fflzj") pod "33e5df03-ba9b-4fef-a44e-d574a98141e1" (UID: "33e5df03-ba9b-4fef-a44e-d574a98141e1"). InnerVolumeSpecName "kube-api-access-fflzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 00:00:12.390353 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.390332    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bead5ba-e3c5-419e-b172-2d35682a9b48-kube-api-access-gfrh5" (OuterVolumeSpecName: "kube-api-access-gfrh5") pod "6bead5ba-e3c5-419e-b172-2d35682a9b48" (UID: "6bead5ba-e3c5-419e-b172-2d35682a9b48"). InnerVolumeSpecName "kube-api-access-gfrh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 00:00:12.488828 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.488804    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fflzj\" (UniqueName: \"kubernetes.io/projected/33e5df03-ba9b-4fef-a44e-d574a98141e1-kube-api-access-fflzj\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:00:12.488828 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.488821    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bead5ba-e3c5-419e-b172-2d35682a9b48-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:00:12.488828 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.488832    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfrh5\" (UniqueName: \"kubernetes.io/projected/6bead5ba-e3c5-419e-b172-2d35682a9b48-kube-api-access-gfrh5\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:00:12.489291 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:12.488840    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33e5df03-ba9b-4fef-a44e-d574a98141e1-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:00:13.030023 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.029986    2564 generic.go:356] "Generic (PLEG): container finished" podID="6bead5ba-e3c5-419e-b172-2d35682a9b48" containerID="b15de6052048c096f124737af856bd45f9caf9993a8b214881f95524c7c0a8ee" exitCode=0
Sep 05 00:00:13.030194 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.030039    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"
Sep 05 00:00:13.030194 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.030073    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf" event={"ID":"6bead5ba-e3c5-419e-b172-2d35682a9b48","Type":"ContainerDied","Data":"b15de6052048c096f124737af856bd45f9caf9993a8b214881f95524c7c0a8ee"}
Sep 05 00:00:13.030194 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.030126    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf" event={"ID":"6bead5ba-e3c5-419e-b172-2d35682a9b48","Type":"ContainerDied","Data":"b474f4fdbe63a5b70b1e1eac015cb6851d9dd215002e5c3cd17bee8b3e1c3b57"}
Sep 05 00:00:13.030194 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.030150    2564 scope.go:117] "RemoveContainer" containerID="b15de6052048c096f124737af856bd45f9caf9993a8b214881f95524c7c0a8ee"
Sep 05 00:00:13.031551 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.031476    2564 generic.go:356] "Generic (PLEG): container finished" podID="33e5df03-ba9b-4fef-a44e-d574a98141e1" containerID="f544892ea5c1ebccacdabc2abb7582387974abe5dbf2982a09d4abe13c75cbc8" exitCode=0
Sep 05 00:00:13.031551 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.031520    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z" event={"ID":"33e5df03-ba9b-4fef-a44e-d574a98141e1","Type":"ContainerDied","Data":"f544892ea5c1ebccacdabc2abb7582387974abe5dbf2982a09d4abe13c75cbc8"}
Sep 05 00:00:13.031551 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.031532    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"
Sep 05 00:00:13.031691 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.031570    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z" event={"ID":"33e5df03-ba9b-4fef-a44e-d574a98141e1","Type":"ContainerDied","Data":"70d4b9f391ec3713adbbc0bade455ccb1af8cce5edd62ab9204410cf08327a43"}
Sep 05 00:00:13.044523 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.044508    2564 scope.go:117] "RemoveContainer" containerID="b15de6052048c096f124737af856bd45f9caf9993a8b214881f95524c7c0a8ee"
Sep 05 00:00:13.044772 ip-10-0-139-127 kubenswrapper[2564]: E0905 00:00:13.044756    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b15de6052048c096f124737af856bd45f9caf9993a8b214881f95524c7c0a8ee\": container with ID starting with b15de6052048c096f124737af856bd45f9caf9993a8b214881f95524c7c0a8ee not found: ID does not exist" containerID="b15de6052048c096f124737af856bd45f9caf9993a8b214881f95524c7c0a8ee"
Sep 05 00:00:13.044809 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.044781    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b15de6052048c096f124737af856bd45f9caf9993a8b214881f95524c7c0a8ee"} err="failed to get container status \"b15de6052048c096f124737af856bd45f9caf9993a8b214881f95524c7c0a8ee\": rpc error: code = NotFound desc = could not find container \"b15de6052048c096f124737af856bd45f9caf9993a8b214881f95524c7c0a8ee\": container with ID starting with b15de6052048c096f124737af856bd45f9caf9993a8b214881f95524c7c0a8ee not found: ID does not exist"
Sep 05 00:00:13.044809 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.044796    2564 scope.go:117] "RemoveContainer" containerID="f544892ea5c1ebccacdabc2abb7582387974abe5dbf2982a09d4abe13c75cbc8"
Sep 05 00:00:13.052248 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.052231    2564 scope.go:117] "RemoveContainer" containerID="f544892ea5c1ebccacdabc2abb7582387974abe5dbf2982a09d4abe13c75cbc8"
Sep 05 00:00:13.052495 ip-10-0-139-127 kubenswrapper[2564]: E0905 00:00:13.052478    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f544892ea5c1ebccacdabc2abb7582387974abe5dbf2982a09d4abe13c75cbc8\": container with ID starting with f544892ea5c1ebccacdabc2abb7582387974abe5dbf2982a09d4abe13c75cbc8 not found: ID does not exist" containerID="f544892ea5c1ebccacdabc2abb7582387974abe5dbf2982a09d4abe13c75cbc8"
Sep 05 00:00:13.052542 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.052502    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f544892ea5c1ebccacdabc2abb7582387974abe5dbf2982a09d4abe13c75cbc8"} err="failed to get container status \"f544892ea5c1ebccacdabc2abb7582387974abe5dbf2982a09d4abe13c75cbc8\": rpc error: code = NotFound desc = could not find container \"f544892ea5c1ebccacdabc2abb7582387974abe5dbf2982a09d4abe13c75cbc8\": container with ID starting with f544892ea5c1ebccacdabc2abb7582387974abe5dbf2982a09d4abe13c75cbc8 not found: ID does not exist"
Sep 05 00:00:13.056750 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.056732    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"]
Sep 05 00:00:13.061395 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.061370    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f7eb8e-964ffb85c-9zg2z"]
Sep 05 00:00:13.071152 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.071133    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"]
Sep 05 00:00:13.075075 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:13.075055    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f7eb8e-7b5cb6b9bb-77mtf"]
Sep 05 00:00:14.179261 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:14.179228    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33e5df03-ba9b-4fef-a44e-d574a98141e1" path="/var/lib/kubelet/pods/33e5df03-ba9b-4fef-a44e-d574a98141e1/volumes"
Sep 05 00:00:14.179663 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:14.179648    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bead5ba-e3c5-419e-b172-2d35682a9b48" path="/var/lib/kubelet/pods/6bead5ba-e3c5-419e-b172-2d35682a9b48/volumes"
Sep 05 00:00:17.767316 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:17.767281    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:17.868719 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:17.868686    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:18.047856 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.047768    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:18.530609 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.530578    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl" podUID="92ef599b-cb26-42aa-b008-2f317bdf2b59" containerName="main" containerID="cri-o://7d37a120ea0cd8b541360c3e5bfdbd1f5112b77f451f4b48bb7ec51f4cbe05e3" gracePeriod=45
Sep 05 00:00:18.786481 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.786422    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 05 00:00:18.947263 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.947231    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-dshm\") pod \"92ef599b-cb26-42aa-b008-2f317bdf2b59\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") "
Sep 05 00:00:18.947263 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.947281    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-model-cache\") pod \"92ef599b-cb26-42aa-b008-2f317bdf2b59\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") "
Sep 05 00:00:18.947439 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.947327    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-tmp-dir\") pod \"92ef599b-cb26-42aa-b008-2f317bdf2b59\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") "
Sep 05 00:00:18.947439 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.947364    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dx6mb\" (UniqueName: \"kubernetes.io/projected/92ef599b-cb26-42aa-b008-2f317bdf2b59-kube-api-access-dx6mb\") pod \"92ef599b-cb26-42aa-b008-2f317bdf2b59\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") "
Sep 05 00:00:18.947439 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.947416    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ef599b-cb26-42aa-b008-2f317bdf2b59-tls-certs\") pod \"92ef599b-cb26-42aa-b008-2f317bdf2b59\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") "
Sep 05 00:00:18.947587 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.947446    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-home\") pod \"92ef599b-cb26-42aa-b008-2f317bdf2b59\" (UID: \"92ef599b-cb26-42aa-b008-2f317bdf2b59\") "
Sep 05 00:00:18.947646 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.947587    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-model-cache" (OuterVolumeSpecName: "model-cache") pod "92ef599b-cb26-42aa-b008-2f317bdf2b59" (UID: "92ef599b-cb26-42aa-b008-2f317bdf2b59"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 00:00:18.947646 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.947602    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "92ef599b-cb26-42aa-b008-2f317bdf2b59" (UID: "92ef599b-cb26-42aa-b008-2f317bdf2b59"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 00:00:18.947798 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.947779    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-model-cache\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:00:18.947850 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.947791    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-home" (OuterVolumeSpecName: "home") pod "92ef599b-cb26-42aa-b008-2f317bdf2b59" (UID: "92ef599b-cb26-42aa-b008-2f317bdf2b59"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 00:00:18.947890 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.947803    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-tmp-dir\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:00:18.949354 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.949332    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-dshm" (OuterVolumeSpecName: "dshm") pod "92ef599b-cb26-42aa-b008-2f317bdf2b59" (UID: "92ef599b-cb26-42aa-b008-2f317bdf2b59"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 00:00:18.949559 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.949541    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92ef599b-cb26-42aa-b008-2f317bdf2b59-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "92ef599b-cb26-42aa-b008-2f317bdf2b59" (UID: "92ef599b-cb26-42aa-b008-2f317bdf2b59"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 00:00:18.949605 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:18.949561    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92ef599b-cb26-42aa-b008-2f317bdf2b59-kube-api-access-dx6mb" (OuterVolumeSpecName: "kube-api-access-dx6mb") pod "92ef599b-cb26-42aa-b008-2f317bdf2b59" (UID: "92ef599b-cb26-42aa-b008-2f317bdf2b59"). InnerVolumeSpecName "kube-api-access-dx6mb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 00:00:19.048605 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:19.048540    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx6mb\" (UniqueName: \"kubernetes.io/projected/92ef599b-cb26-42aa-b008-2f317bdf2b59-kube-api-access-dx6mb\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:00:19.048605 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:19.048573    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92ef599b-cb26-42aa-b008-2f317bdf2b59-tls-certs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:00:19.048605 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:19.048583    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-home\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:00:19.048605 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:19.048590    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92ef599b-cb26-42aa-b008-2f317bdf2b59-dshm\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:00:19.075101 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:19.075071    2564 generic.go:356] "Generic (PLEG): container finished" podID="92ef599b-cb26-42aa-b008-2f317bdf2b59" containerID="7d37a120ea0cd8b541360c3e5bfdbd1f5112b77f451f4b48bb7ec51f4cbe05e3" exitCode=0
Sep 05 00:00:19.075214 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:19.075123    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"
Sep 05 00:00:19.075214 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:19.075156    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl" event={"ID":"92ef599b-cb26-42aa-b008-2f317bdf2b59","Type":"ContainerDied","Data":"7d37a120ea0cd8b541360c3e5bfdbd1f5112b77f451f4b48bb7ec51f4cbe05e3"}
Sep 05 00:00:19.075214 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:19.075204    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl" event={"ID":"92ef599b-cb26-42aa-b008-2f317bdf2b59","Type":"ContainerDied","Data":"ede3e78ea85c390f5ed9a804f8914f106d603b04941933b48138379d22968c29"}
Sep 05 00:00:19.075375 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:19.075223    2564 scope.go:117] "RemoveContainer" containerID="7d37a120ea0cd8b541360c3e5bfdbd1f5112b77f451f4b48bb7ec51f4cbe05e3"
Sep 05 00:00:19.084054 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:19.084036    2564 scope.go:117] "RemoveContainer" containerID="7d37a120ea0cd8b541360c3e5bfdbd1f5112b77f451f4b48bb7ec51f4cbe05e3"
Sep 05 00:00:19.084330 ip-10-0-139-127 kubenswrapper[2564]: E0905 00:00:19.084307    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d37a120ea0cd8b541360c3e5bfdbd1f5112b77f451f4b48bb7ec51f4cbe05e3\": container with ID starting with 7d37a120ea0cd8b541360c3e5bfdbd1f5112b77f451f4b48bb7ec51f4cbe05e3 not found: ID does not exist" containerID="7d37a120ea0cd8b541360c3e5bfdbd1f5112b77f451f4b48bb7ec51f4cbe05e3"
Sep 05 00:00:19.084418 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:19.084337    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d37a120ea0cd8b541360c3e5bfdbd1f5112b77f451f4b48bb7ec51f4cbe05e3"} err="failed to get container status \"7d37a120ea0cd8b541360c3e5bfdbd1f5112b77f451f4b48bb7ec51f4cbe05e3\": rpc error: code = NotFound desc = could not find container \"7d37a120ea0cd8b541360c3e5bfdbd1f5112b77f451f4b48bb7ec51f4cbe05e3\": container with ID starting with 7d37a120ea0cd8b541360c3e5bfdbd1f5112b77f451f4b48bb7ec51f4cbe05e3 not found: ID does not exist"
Sep 05 00:00:19.095354 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:19.095327    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"]
Sep 05 00:00:19.098937 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:19.098892    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5245c1/rate-test-model-5245c1-kserve-bcf4967b7-hnqtl"]
Sep 05 00:00:20.046747 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:20.046705    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:20.100090 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:20.100057    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:20.174973 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:20.174943    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92ef599b-cb26-42aa-b008-2f317bdf2b59" path="/var/lib/kubelet/pods/92ef599b-cb26-42aa-b008-2f317bdf2b59/volumes"
Sep 05 00:00:33.736282 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:33.736243    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:35.121209 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:35.121169    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:35.221332 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:35.221304    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:37.880724 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:37.880681    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:37.974961 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:37.974919    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:38.978869 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.978836    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"]
Sep 05 00:00:38.979410 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979390    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bead5ba-e3c5-419e-b172-2d35682a9b48" containerName="payload-pre-processing"
Sep 05 00:00:38.979494 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979413    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bead5ba-e3c5-419e-b172-2d35682a9b48" containerName="payload-pre-processing"
Sep 05 00:00:38.979494 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979423    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80df0ae5-caec-4846-963e-053de587bb78" containerName="cleanup"
Sep 05 00:00:38.979494 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979431    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="80df0ae5-caec-4846-963e-053de587bb78" containerName="cleanup"
Sep 05 00:00:38.979494 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979439    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92ef599b-cb26-42aa-b008-2f317bdf2b59" containerName="main"
Sep 05 00:00:38.979494 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979447    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="92ef599b-cb26-42aa-b008-2f317bdf2b59" containerName="main"
Sep 05 00:00:38.979494 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979456    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0" containerName="revoke-keys"
Sep 05 00:00:38.979494 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979465    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0" containerName="revoke-keys"
Sep 05 00:00:38.979494 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979479    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f570bdc0-e8e4-4ecc-9cd9-fef771da8e9f" containerName="cleanup"
Sep 05 00:00:38.979494 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979487    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f570bdc0-e8e4-4ecc-9cd9-fef771da8e9f" containerName="cleanup"
Sep 05 00:00:38.979970 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979512    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e5df03-ba9b-4fef-a44e-d574a98141e1" containerName="payload-processing"
Sep 05 00:00:38.979970 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979521    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e5df03-ba9b-4fef-a44e-d574a98141e1" containerName="payload-processing"
Sep 05 00:00:38.979970 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979605    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="80df0ae5-caec-4846-963e-053de587bb78" containerName="cleanup"
Sep 05 00:00:38.979970 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979618    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bead5ba-e3c5-419e-b172-2d35682a9b48" containerName="payload-pre-processing"
Sep 05 00:00:38.979970 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979628    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="92ef599b-cb26-42aa-b008-2f317bdf2b59" containerName="main"
Sep 05 00:00:38.979970 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979639    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a13a2dd9-a5d0-482d-bbc6-bbc9feeaf7f0" containerName="revoke-keys"
Sep 05 00:00:38.979970 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979649    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f570bdc0-e8e4-4ecc-9cd9-fef771da8e9f" containerName="cleanup"
Sep 05 00:00:38.979970 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.979657    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e5df03-ba9b-4fef-a44e-d574a98141e1" containerName="payload-processing"
Sep 05 00:00:38.983289 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.983265    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:38.986024 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.986002    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-64702c-serving-cert\""
Sep 05 00:00:38.987254 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.987227    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 00:00:38.987369 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.987275    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 05 00:00:38.987369 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.987285    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 05 00:00:38.987369 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.987275    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mnd5p\""
Sep 05 00:00:38.990843 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:38.990771    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"]
Sep 05 00:00:39.042379 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.042307    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:39.116450 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.116423    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lx4q\" (UniqueName: \"kubernetes.io/projected/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-kube-api-access-9lx4q\") pod \"maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:39.116563 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.116473    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-maas-api-tls\") pod \"maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:39.116563 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.116532    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:39.116634 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.116573    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-metrics-certs\") pod \"maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:39.143591 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.143562    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:39.216994 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.216971    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lx4q\" (UniqueName: \"kubernetes.io/projected/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-kube-api-access-9lx4q\") pod \"maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:39.217122 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.217047    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-maas-api-tls\") pod \"maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:39.217122 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.217099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:39.217231 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.217145    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-metrics-certs\") pod \"maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:39.217740 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.217716    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:39.219548 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.219527    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-metrics-certs\") pod \"maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:39.219790 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.219773    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-maas-api-tls\") pod \"maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:39.225853 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.225831    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lx4q\" (UniqueName: \"kubernetes.io/projected/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-kube-api-access-9lx4q\") pod \"maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:39.295260 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.295200    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:39.494997 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.494962    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:39.572325 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.572248    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"]
Sep 05 00:00:39.574608 ip-10-0-139-127 kubenswrapper[2564]: W0905 00:00:39.574574    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64851d1b_dc9a_4d7a_8ba2_a803396da2fe.slice/crio-8773e0024e62fd8022fa588c7dc674d7d7a4f82a56748e50da703917cb809274 WatchSource:0}: Error finding container 8773e0024e62fd8022fa588c7dc674d7d7a4f82a56748e50da703917cb809274: Status 404 returned error can't find the container with id 8773e0024e62fd8022fa588c7dc674d7d7a4f82a56748e50da703917cb809274
Sep 05 00:00:39.574756 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.574645    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:00:39.575916 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.575883    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 00:00:39.596252 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.596225    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"]
Sep 05 00:00:39.600999 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.600984    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"
Sep 05 00:00:39.610197 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.610177    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-64702c-dockercfg-qmpd7\""
Sep 05 00:00:39.610306 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.610181    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-64702c\""
Sep 05 00:00:39.613200 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.613179    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"]
Sep 05 00:00:39.620558 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.620535    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"]
Sep 05 00:00:39.620867 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.620847    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdf63246-b183-40e3-9cc6-95ef6de59b10-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27\" (UID: \"bdf63246-b183-40e3-9cc6-95ef6de59b10\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"
Sep 05 00:00:39.620936 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.620880    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftzlk\" (UniqueName: \"kubernetes.io/projected/bdf63246-b183-40e3-9cc6-95ef6de59b10-kube-api-access-ftzlk\") pod \"payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27\" (UID: \"bdf63246-b183-40e3-9cc6-95ef6de59b10\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"
Sep 05 00:00:39.624650 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.624633    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"
Sep 05 00:00:39.635237 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.635206    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"]
Sep 05 00:00:39.722496 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.722469    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/760c36ca-6e89-446e-a69b-dbcc57721ba0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd\" (UID: \"760c36ca-6e89-446e-a69b-dbcc57721ba0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"
Sep 05 00:00:39.722629 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.722503    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdf63246-b183-40e3-9cc6-95ef6de59b10-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27\" (UID: \"bdf63246-b183-40e3-9cc6-95ef6de59b10\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"
Sep 05 00:00:39.722629 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.722528    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftzlk\" (UniqueName: \"kubernetes.io/projected/bdf63246-b183-40e3-9cc6-95ef6de59b10-kube-api-access-ftzlk\") pod \"payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27\" (UID: \"bdf63246-b183-40e3-9cc6-95ef6de59b10\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"
Sep 05 00:00:39.722629 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.722598    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh2wk\" (UniqueName: \"kubernetes.io/projected/760c36ca-6e89-446e-a69b-dbcc57721ba0-kube-api-access-fh2wk\") pod \"payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd\" (UID: \"760c36ca-6e89-446e-a69b-dbcc57721ba0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"
Sep 05 00:00:39.723176 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.723160    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdf63246-b183-40e3-9cc6-95ef6de59b10-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27\" (UID: \"bdf63246-b183-40e3-9cc6-95ef6de59b10\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"
Sep 05 00:00:39.730443 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.730426    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftzlk\" (UniqueName: \"kubernetes.io/projected/bdf63246-b183-40e3-9cc6-95ef6de59b10-kube-api-access-ftzlk\") pod \"payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27\" (UID: \"bdf63246-b183-40e3-9cc6-95ef6de59b10\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"
Sep 05 00:00:39.823160 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.823098    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/760c36ca-6e89-446e-a69b-dbcc57721ba0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd\" (UID: \"760c36ca-6e89-446e-a69b-dbcc57721ba0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"
Sep 05 00:00:39.823160 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.823156    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh2wk\" (UniqueName: \"kubernetes.io/projected/760c36ca-6e89-446e-a69b-dbcc57721ba0-kube-api-access-fh2wk\") pod \"payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd\" (UID: \"760c36ca-6e89-446e-a69b-dbcc57721ba0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"
Sep 05 00:00:39.823767 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.823738    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/760c36ca-6e89-446e-a69b-dbcc57721ba0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd\" (UID: \"760c36ca-6e89-446e-a69b-dbcc57721ba0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"
Sep 05 00:00:39.831148 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.831121    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh2wk\" (UniqueName: \"kubernetes.io/projected/760c36ca-6e89-446e-a69b-dbcc57721ba0-kube-api-access-fh2wk\") pod \"payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd\" (UID: \"760c36ca-6e89-446e-a69b-dbcc57721ba0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"
Sep 05 00:00:39.911775 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.911749    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"
Sep 05 00:00:39.935459 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:39.935440    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"
Sep 05 00:00:40.183777 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:40.183744    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std" event={"ID":"64851d1b-dc9a-4d7a-8ba2-a803396da2fe","Type":"ContainerStarted","Data":"8773e0024e62fd8022fa588c7dc674d7d7a4f82a56748e50da703917cb809274"}
Sep 05 00:00:40.184250 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:40.183830    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"]
Sep 05 00:00:40.188076 ip-10-0-139-127 kubenswrapper[2564]: W0905 00:00:40.188049    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdf63246_b183_40e3_9cc6_95ef6de59b10.slice/crio-190fb7a247fa45c8afef5857ba5c74b961128c5100c88aedb00aac825678a3a1 WatchSource:0}: Error finding container 190fb7a247fa45c8afef5857ba5c74b961128c5100c88aedb00aac825678a3a1: Status 404 returned error can't find the container with id 190fb7a247fa45c8afef5857ba5c74b961128c5100c88aedb00aac825678a3a1
Sep 05 00:00:40.220097 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:40.219922    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"]
Sep 05 00:00:40.221963 ip-10-0-139-127 kubenswrapper[2564]: W0905 00:00:40.221933    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod760c36ca_6e89_446e_a69b_dbcc57721ba0.slice/crio-31809bae956680e3fc93477e989a6f8429122c8e014275a98b98ad6b6e9b80f2 WatchSource:0}: Error finding container 31809bae956680e3fc93477e989a6f8429122c8e014275a98b98ad6b6e9b80f2: Status 404 returned error can't find the container with id 31809bae956680e3fc93477e989a6f8429122c8e014275a98b98ad6b6e9b80f2
Sep 05 00:00:41.192281 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:41.192245    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd" event={"ID":"760c36ca-6e89-446e-a69b-dbcc57721ba0","Type":"ContainerStarted","Data":"5c820e62659dcbbc798b76f5f591a303ce64647c0a2c7082a24d989481ef68ec"}
Sep 05 00:00:41.192728 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:41.192308    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd" event={"ID":"760c36ca-6e89-446e-a69b-dbcc57721ba0","Type":"ContainerStarted","Data":"31809bae956680e3fc93477e989a6f8429122c8e014275a98b98ad6b6e9b80f2"}
Sep 05 00:00:41.192728 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:41.192340    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"
Sep 05 00:00:41.194379 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:41.194296    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27" event={"ID":"bdf63246-b183-40e3-9cc6-95ef6de59b10","Type":"ContainerStarted","Data":"650074e7e9014b7da171668e959a927d18a13ebd2a81a022ef8b4cc0d4725bda"}
Sep 05 00:00:41.194379 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:41.194348    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27" event={"ID":"bdf63246-b183-40e3-9cc6-95ef6de59b10","Type":"ContainerStarted","Data":"190fb7a247fa45c8afef5857ba5c74b961128c5100c88aedb00aac825678a3a1"}
Sep 05 00:00:41.194575 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:41.194424    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"
Sep 05 00:00:41.208641 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:41.208590    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd" podStartSLOduration=2.208574683 podStartE2EDuration="2.208574683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 00:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 00:00:41.207501809 +0000 UTC m=+1429.455502321" watchObservedRunningTime="2026-09-05 00:00:41.208574683 +0000 UTC m=+1429.456575196"
Sep 05 00:00:41.223476 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:41.223438    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27" podStartSLOduration=2.223424859 podStartE2EDuration="2.223424859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 00:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 00:00:41.222408382 +0000 UTC m=+1429.470408923" watchObservedRunningTime="2026-09-05 00:00:41.223424859 +0000 UTC m=+1429.471425374"
Sep 05 00:00:43.211133 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:43.211101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std" event={"ID":"64851d1b-dc9a-4d7a-8ba2-a803396da2fe","Type":"ContainerStarted","Data":"c057694e7646367515b3b42d117f2e286877d7e31c60e2c9d76dc03e6d872f10"}
Sep 05 00:00:43.211522 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:43.211241    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:43.230720 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:43.230672    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std" podStartSLOduration=2.683231394 podStartE2EDuration="5.23065852s" totalImagesPullingTime="2.547427126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 00:00:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 00:00:43.228529523 +0000 UTC m=+1431.476530059" watchObservedRunningTime="2026-09-05 00:00:43.23065852 +0000 UTC m=+1431.478659046"
Sep 05 00:00:49.221256 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:49.221228    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:00:52.206262 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:52.206230    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"
Sep 05 00:00:52.206630 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:52.206292    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"
Sep 05 00:00:59.955655 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:00:59.955618    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:00.760112 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:00.760079    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:02.926282 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:02.926248    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:08.048402 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:08.048361    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd_760c36ca-6e89-446e-a69b-dbcc57721ba0/payload-processing/0.log"
Sep 05 00:01:13.508974 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:13.508928    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:41.121344 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:41.121310    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd_760c36ca-6e89-446e-a69b-dbcc57721ba0/payload-processing/0.log"
Sep 05 00:01:41.402128 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:41.402042    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:41.723466 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:41.723380    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:41.926836 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:41.926803    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:44.532595 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:44.532550    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:44.633188 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:44.633152    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:45.565961 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.565921    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"]
Sep 05 00:01:45.569719 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.569696    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:45.572400 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.572378    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-b3c355-serving-cert\""
Sep 05 00:01:45.580185 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.580161    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"]
Sep 05 00:01:45.600873 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.600841    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:45.657851 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.657831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:45.725983 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.725962    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a578293c-603e-4465-8893-8802bf70f816-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:45.726075 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.725990    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a578293c-603e-4465-8893-8802bf70f816-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:45.726075 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.726013    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a578293c-603e-4465-8893-8802bf70f816-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:45.726146 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.726124    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdxxt\" (UniqueName: \"kubernetes.io/projected/a578293c-603e-4465-8893-8802bf70f816-kube-api-access-fdxxt\") pod \"maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:45.788941 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.788876    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:45.826792 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.826726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdxxt\" (UniqueName: \"kubernetes.io/projected/a578293c-603e-4465-8893-8802bf70f816-kube-api-access-fdxxt\") pod \"maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:45.826932 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.826843    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a578293c-603e-4465-8893-8802bf70f816-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:45.826932 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.826872    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a578293c-603e-4465-8893-8802bf70f816-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:45.827050 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.826929    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a578293c-603e-4465-8893-8802bf70f816-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:45.827574 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.827556    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a578293c-603e-4465-8893-8802bf70f816-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:45.829377 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.829351    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a578293c-603e-4465-8893-8802bf70f816-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:45.829463 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.829382    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a578293c-603e-4465-8893-8802bf70f816-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:45.834368 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.834350    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdxxt\" (UniqueName: \"kubernetes.io/projected/a578293c-603e-4465-8893-8802bf70f816-kube-api-access-fdxxt\") pod \"maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:45.881434 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:45.881410    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:46.151579 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.151546    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"]
Sep 05 00:01:46.153861 ip-10-0-139-127 kubenswrapper[2564]: W0905 00:01:46.153829    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda578293c_603e_4465_8893_8802bf70f816.slice/crio-45a4ab77efec702c51b48cbbc95fc7dd1aabf4ba61e61f94a596733266b44244 WatchSource:0}: Error finding container 45a4ab77efec702c51b48cbbc95fc7dd1aabf4ba61e61f94a596733266b44244: Status 404 returned error can't find the container with id 45a4ab77efec702c51b48cbbc95fc7dd1aabf4ba61e61f94a596733266b44244
Sep 05 00:01:46.154238 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.154219    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"]
Sep 05 00:01:46.159528 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.159507    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"
Sep 05 00:01:46.162527 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.162483    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-b3c355\""
Sep 05 00:01:46.162787 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.162764    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-b3c355-dockercfg-n57v4\""
Sep 05 00:01:46.176008 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.175986    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"]
Sep 05 00:01:46.331435 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.331403    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kv4x\" (UniqueName: \"kubernetes.io/projected/bafd86d2-353d-419c-8bee-522c47912a4e-kube-api-access-7kv4x\") pod \"payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm\" (UID: \"bafd86d2-353d-419c-8bee-522c47912a4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"
Sep 05 00:01:46.331588 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.331454    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bafd86d2-353d-419c-8bee-522c47912a4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm\" (UID: \"bafd86d2-353d-419c-8bee-522c47912a4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"
Sep 05 00:01:46.432778 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.432705    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kv4x\" (UniqueName: \"kubernetes.io/projected/bafd86d2-353d-419c-8bee-522c47912a4e-kube-api-access-7kv4x\") pod \"payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm\" (UID: \"bafd86d2-353d-419c-8bee-522c47912a4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"
Sep 05 00:01:46.432778 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.432768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bafd86d2-353d-419c-8bee-522c47912a4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm\" (UID: \"bafd86d2-353d-419c-8bee-522c47912a4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"
Sep 05 00:01:46.433466 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.433447    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bafd86d2-353d-419c-8bee-522c47912a4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm\" (UID: \"bafd86d2-353d-419c-8bee-522c47912a4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"
Sep 05 00:01:46.440014 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.439993    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kv4x\" (UniqueName: \"kubernetes.io/projected/bafd86d2-353d-419c-8bee-522c47912a4e-kube-api-access-7kv4x\") pod \"payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm\" (UID: \"bafd86d2-353d-419c-8bee-522c47912a4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"
Sep 05 00:01:46.473254 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.473234    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"
Sep 05 00:01:46.545228 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.545197    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf" event={"ID":"a578293c-603e-4465-8893-8802bf70f816","Type":"ContainerStarted","Data":"45a4ab77efec702c51b48cbbc95fc7dd1aabf4ba61e61f94a596733266b44244"}
Sep 05 00:01:46.748062 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:46.748035    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"]
Sep 05 00:01:46.749570 ip-10-0-139-127 kubenswrapper[2564]: W0905 00:01:46.749543    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbafd86d2_353d_419c_8bee_522c47912a4e.slice/crio-1935ebb9ebd7c24abf8094780c687b32e9c6ca62a13b97da3405f67b052a7d92 WatchSource:0}: Error finding container 1935ebb9ebd7c24abf8094780c687b32e9c6ca62a13b97da3405f67b052a7d92: Status 404 returned error can't find the container with id 1935ebb9ebd7c24abf8094780c687b32e9c6ca62a13b97da3405f67b052a7d92
Sep 05 00:01:47.552703 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:47.552666    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm" event={"ID":"bafd86d2-353d-419c-8bee-522c47912a4e","Type":"ContainerStarted","Data":"895e038ffb2264d2b69476bf6c1105045b854716dc7fb422aec1dc4b995d17fc"}
Sep 05 00:01:47.552917 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:47.552728    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm" event={"ID":"bafd86d2-353d-419c-8bee-522c47912a4e","Type":"ContainerStarted","Data":"1935ebb9ebd7c24abf8094780c687b32e9c6ca62a13b97da3405f67b052a7d92"}
Sep 05 00:01:47.552917 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:47.552773    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"
Sep 05 00:01:47.554314 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:47.554290    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf" event={"ID":"a578293c-603e-4465-8893-8802bf70f816","Type":"ContainerStarted","Data":"d428a1ed9fd60cf84ddbf15c99b1e89fee4b66b850822d8e1641a75de7842e89"}
Sep 05 00:01:47.554432 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:47.554338    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:47.568835 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:47.568787    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm" podStartSLOduration=1.568771286 podStartE2EDuration="1.568771286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 00:01:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 00:01:47.566849291 +0000 UTC m=+1495.814849825" watchObservedRunningTime="2026-09-05 00:01:47.568771286 +0000 UTC m=+1495.816771798"
Sep 05 00:01:47.571737 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:47.571689    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:47.584068 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:47.584027    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf" podStartSLOduration=2.26121915 podStartE2EDuration="2.584015574s" totalImagesPullingTime="322.796424ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 00:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 00:01:47.582320091 +0000 UTC m=+1495.830320633" watchObservedRunningTime="2026-09-05 00:01:47.584015574 +0000 UTC m=+1495.832016119"
Sep 05 00:01:47.624483 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:47.624433    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:01:53.564090 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:53.564061    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:01:58.561067 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:01:58.561035    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"
Sep 05 00:02:09.288439 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:09.288402    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:16.079047 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.077711    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"]
Sep 05 00:02:16.079999 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.079943    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf" podUID="a578293c-603e-4465-8893-8802bf70f816" containerName="maas-api" containerID="cri-o://d428a1ed9fd60cf84ddbf15c99b1e89fee4b66b850822d8e1641a75de7842e89" gracePeriod=30
Sep 05 00:02:16.106689 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.106650    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:16.135826 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.135795    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"]
Sep 05 00:02:16.136098 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.136057    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm" podUID="bafd86d2-353d-419c-8bee-522c47912a4e" containerName="payload-pre-processing" containerID="cri-o://895e038ffb2264d2b69476bf6c1105045b854716dc7fb422aec1dc4b995d17fc" gracePeriod=30
Sep 05 00:02:16.206839 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.206800    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:16.282136 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.282051    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:16.339729 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.339630    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:16.389273 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.389251    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:02:16.416786 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.416759    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"
Sep 05 00:02:16.509530 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.509495    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bafd86d2-353d-419c-8bee-522c47912a4e-plugins-config-volume\") pod \"bafd86d2-353d-419c-8bee-522c47912a4e\" (UID: \"bafd86d2-353d-419c-8bee-522c47912a4e\") "
Sep 05 00:02:16.509708 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.509564    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a578293c-603e-4465-8893-8802bf70f816-openshift-service-ca\") pod \"a578293c-603e-4465-8893-8802bf70f816\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") "
Sep 05 00:02:16.509708 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.509608    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kv4x\" (UniqueName: \"kubernetes.io/projected/bafd86d2-353d-419c-8bee-522c47912a4e-kube-api-access-7kv4x\") pod \"bafd86d2-353d-419c-8bee-522c47912a4e\" (UID: \"bafd86d2-353d-419c-8bee-522c47912a4e\") "
Sep 05 00:02:16.509708 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.509647    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a578293c-603e-4465-8893-8802bf70f816-maas-api-tls\") pod \"a578293c-603e-4465-8893-8802bf70f816\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") "
Sep 05 00:02:16.509708 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.509692    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a578293c-603e-4465-8893-8802bf70f816-metrics-certs\") pod \"a578293c-603e-4465-8893-8802bf70f816\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") "
Sep 05 00:02:16.509958 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.509710    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdxxt\" (UniqueName: \"kubernetes.io/projected/a578293c-603e-4465-8893-8802bf70f816-kube-api-access-fdxxt\") pod \"a578293c-603e-4465-8893-8802bf70f816\" (UID: \"a578293c-603e-4465-8893-8802bf70f816\") "
Sep 05 00:02:16.509958 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.509831    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bafd86d2-353d-419c-8bee-522c47912a4e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bafd86d2-353d-419c-8bee-522c47912a4e" (UID: "bafd86d2-353d-419c-8bee-522c47912a4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 00:02:16.509958 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.509947    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a578293c-603e-4465-8893-8802bf70f816-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a578293c-603e-4465-8893-8802bf70f816" (UID: "a578293c-603e-4465-8893-8802bf70f816"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 00:02:16.510130 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.510081    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bafd86d2-353d-419c-8bee-522c47912a4e-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:02:16.510130 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.510102    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a578293c-603e-4465-8893-8802bf70f816-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:02:16.512054 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.512021    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bafd86d2-353d-419c-8bee-522c47912a4e-kube-api-access-7kv4x" (OuterVolumeSpecName: "kube-api-access-7kv4x") pod "bafd86d2-353d-419c-8bee-522c47912a4e" (UID: "bafd86d2-353d-419c-8bee-522c47912a4e"). InnerVolumeSpecName "kube-api-access-7kv4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 00:02:16.512190 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.512111    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a578293c-603e-4465-8893-8802bf70f816-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a578293c-603e-4465-8893-8802bf70f816" (UID: "a578293c-603e-4465-8893-8802bf70f816"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 00:02:16.512190 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.512132    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a578293c-603e-4465-8893-8802bf70f816-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a578293c-603e-4465-8893-8802bf70f816" (UID: "a578293c-603e-4465-8893-8802bf70f816"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 00:02:16.512301 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.512237    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a578293c-603e-4465-8893-8802bf70f816-kube-api-access-fdxxt" (OuterVolumeSpecName: "kube-api-access-fdxxt") pod "a578293c-603e-4465-8893-8802bf70f816" (UID: "a578293c-603e-4465-8893-8802bf70f816"). InnerVolumeSpecName "kube-api-access-fdxxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 00:02:16.611001 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.610970    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kv4x\" (UniqueName: \"kubernetes.io/projected/bafd86d2-353d-419c-8bee-522c47912a4e-kube-api-access-7kv4x\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:02:16.611001 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.610999    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a578293c-603e-4465-8893-8802bf70f816-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:02:16.611187 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.611007    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a578293c-603e-4465-8893-8802bf70f816-metrics-certs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:02:16.611187 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.611018    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdxxt\" (UniqueName: \"kubernetes.io/projected/a578293c-603e-4465-8893-8802bf70f816-kube-api-access-fdxxt\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:02:16.643873 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.643836    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:16.719705 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.719673    2564 generic.go:356] "Generic (PLEG): container finished" podID="bafd86d2-353d-419c-8bee-522c47912a4e" containerID="895e038ffb2264d2b69476bf6c1105045b854716dc7fb422aec1dc4b995d17fc" exitCode=0
Sep 05 00:02:16.719881 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.719720    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"
Sep 05 00:02:16.719881 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.719765    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm" event={"ID":"bafd86d2-353d-419c-8bee-522c47912a4e","Type":"ContainerDied","Data":"895e038ffb2264d2b69476bf6c1105045b854716dc7fb422aec1dc4b995d17fc"}
Sep 05 00:02:16.719881 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.719826    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm" event={"ID":"bafd86d2-353d-419c-8bee-522c47912a4e","Type":"ContainerDied","Data":"1935ebb9ebd7c24abf8094780c687b32e9c6ca62a13b97da3405f67b052a7d92"}
Sep 05 00:02:16.719881 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.719859    2564 scope.go:117] "RemoveContainer" containerID="895e038ffb2264d2b69476bf6c1105045b854716dc7fb422aec1dc4b995d17fc"
Sep 05 00:02:16.721239 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.721216    2564 generic.go:356] "Generic (PLEG): container finished" podID="a578293c-603e-4465-8893-8802bf70f816" containerID="d428a1ed9fd60cf84ddbf15c99b1e89fee4b66b850822d8e1641a75de7842e89" exitCode=0
Sep 05 00:02:16.721357 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.721247    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf" event={"ID":"a578293c-603e-4465-8893-8802bf70f816","Type":"ContainerDied","Data":"d428a1ed9fd60cf84ddbf15c99b1e89fee4b66b850822d8e1641a75de7842e89"}
Sep 05 00:02:16.721357 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.721272    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"
Sep 05 00:02:16.721357 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.721292    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf" event={"ID":"a578293c-603e-4465-8893-8802bf70f816","Type":"ContainerDied","Data":"45a4ab77efec702c51b48cbbc95fc7dd1aabf4ba61e61f94a596733266b44244"}
Sep 05 00:02:16.730135 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.730118    2564 scope.go:117] "RemoveContainer" containerID="895e038ffb2264d2b69476bf6c1105045b854716dc7fb422aec1dc4b995d17fc"
Sep 05 00:02:16.730403 ip-10-0-139-127 kubenswrapper[2564]: E0905 00:02:16.730382    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"895e038ffb2264d2b69476bf6c1105045b854716dc7fb422aec1dc4b995d17fc\": container with ID starting with 895e038ffb2264d2b69476bf6c1105045b854716dc7fb422aec1dc4b995d17fc not found: ID does not exist" containerID="895e038ffb2264d2b69476bf6c1105045b854716dc7fb422aec1dc4b995d17fc"
Sep 05 00:02:16.730475 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.730410    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"895e038ffb2264d2b69476bf6c1105045b854716dc7fb422aec1dc4b995d17fc"} err="failed to get container status \"895e038ffb2264d2b69476bf6c1105045b854716dc7fb422aec1dc4b995d17fc\": rpc error: code = NotFound desc = could not find container \"895e038ffb2264d2b69476bf6c1105045b854716dc7fb422aec1dc4b995d17fc\": container with ID starting with 895e038ffb2264d2b69476bf6c1105045b854716dc7fb422aec1dc4b995d17fc not found: ID does not exist"
Sep 05 00:02:16.730475 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.730428    2564 scope.go:117] "RemoveContainer" containerID="d428a1ed9fd60cf84ddbf15c99b1e89fee4b66b850822d8e1641a75de7842e89"
Sep 05 00:02:16.738496 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.738476    2564 scope.go:117] "RemoveContainer" containerID="d428a1ed9fd60cf84ddbf15c99b1e89fee4b66b850822d8e1641a75de7842e89"
Sep 05 00:02:16.738749 ip-10-0-139-127 kubenswrapper[2564]: E0905 00:02:16.738729    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d428a1ed9fd60cf84ddbf15c99b1e89fee4b66b850822d8e1641a75de7842e89\": container with ID starting with d428a1ed9fd60cf84ddbf15c99b1e89fee4b66b850822d8e1641a75de7842e89 not found: ID does not exist" containerID="d428a1ed9fd60cf84ddbf15c99b1e89fee4b66b850822d8e1641a75de7842e89"
Sep 05 00:02:16.738812 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.738759    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d428a1ed9fd60cf84ddbf15c99b1e89fee4b66b850822d8e1641a75de7842e89"} err="failed to get container status \"d428a1ed9fd60cf84ddbf15c99b1e89fee4b66b850822d8e1641a75de7842e89\": rpc error: code = NotFound desc = could not find container \"d428a1ed9fd60cf84ddbf15c99b1e89fee4b66b850822d8e1641a75de7842e89\": container with ID starting with d428a1ed9fd60cf84ddbf15c99b1e89fee4b66b850822d8e1641a75de7842e89 not found: ID does not exist"
Sep 05 00:02:16.741955 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.741886    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"]
Sep 05 00:02:16.746548 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.746526    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b3c355-794b978dbb-zmpqm"]
Sep 05 00:02:16.756046 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.756027    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"]
Sep 05 00:02:16.759303 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:16.759283    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b3c355-6456fbdc6f-fdmtf"]
Sep 05 00:02:18.175265 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:18.175228    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a578293c-603e-4465-8893-8802bf70f816" path="/var/lib/kubelet/pods/a578293c-603e-4465-8893-8802bf70f816/volumes"
Sep 05 00:02:18.175658 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:18.175642    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bafd86d2-353d-419c-8bee-522c47912a4e" path="/var/lib/kubelet/pods/bafd86d2-353d-419c-8bee-522c47912a4e/volumes"
Sep 05 00:02:22.230237 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:22.230198    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:22.331613 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:22.331582    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:22.488334 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:22.488261    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:26.200218 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:26.200179    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:41.509704 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:41.509658    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:41.589782 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:41.589735    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:43.176291 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:43.176257    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:43.276393 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:43.276355    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:45.853234 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:45.853203    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:47.234567 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:47.234533    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:02:49.118665 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:02:49.118628    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:23.480217 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:23.480180    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:24.323772 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:24.323737    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:29.710707 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:29.710671    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:29.810976 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:29.810948    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:29.977226 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:29.977152    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:30.490205 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:30.490164    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:40.774498 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:40.774466    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"]
Sep 05 00:03:40.774978 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:40.774702    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std" podUID="64851d1b-dc9a-4d7a-8ba2-a803396da2fe" containerName="maas-api" containerID="cri-o://c057694e7646367515b3b42d117f2e286877d7e31c60e2c9d76dc03e6d872f10" gracePeriod=30
Sep 05 00:03:40.798792 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:40.798757    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:40.821959 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:40.821930    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"]
Sep 05 00:03:40.822196 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:40.822168    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd" podUID="760c36ca-6e89-446e-a69b-dbcc57721ba0" containerName="payload-processing" containerID="cri-o://5c820e62659dcbbc798b76f5f591a303ce64647c0a2c7082a24d989481ef68ec" gracePeriod=30
Sep 05 00:03:40.825425 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:40.825401    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"]
Sep 05 00:03:40.825663 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:40.825634    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27" podUID="bdf63246-b183-40e3-9cc6-95ef6de59b10" containerName="payload-pre-processing" containerID="cri-o://650074e7e9014b7da171668e959a927d18a13ebd2a81a022ef8b4cc0d4725bda" gracePeriod=30
Sep 05 00:03:40.859058 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:40.859032    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:41.086201 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.086170    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:03:41.115584 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.114339    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"
Sep 05 00:03:41.126521 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.126489    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"
Sep 05 00:03:41.143707 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.143675    2564 generic.go:356] "Generic (PLEG): container finished" podID="64851d1b-dc9a-4d7a-8ba2-a803396da2fe" containerID="c057694e7646367515b3b42d117f2e286877d7e31c60e2c9d76dc03e6d872f10" exitCode=0
Sep 05 00:03:41.143837 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.143720    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"
Sep 05 00:03:41.143837 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.143752    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std" event={"ID":"64851d1b-dc9a-4d7a-8ba2-a803396da2fe","Type":"ContainerDied","Data":"c057694e7646367515b3b42d117f2e286877d7e31c60e2c9d76dc03e6d872f10"}
Sep 05 00:03:41.143993 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.143835    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std" event={"ID":"64851d1b-dc9a-4d7a-8ba2-a803396da2fe","Type":"ContainerDied","Data":"8773e0024e62fd8022fa588c7dc674d7d7a4f82a56748e50da703917cb809274"}
Sep 05 00:03:41.143993 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.143866    2564 scope.go:117] "RemoveContainer" containerID="c057694e7646367515b3b42d117f2e286877d7e31c60e2c9d76dc03e6d872f10"
Sep 05 00:03:41.145240 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.145219    2564 generic.go:356] "Generic (PLEG): container finished" podID="bdf63246-b183-40e3-9cc6-95ef6de59b10" containerID="650074e7e9014b7da171668e959a927d18a13ebd2a81a022ef8b4cc0d4725bda" exitCode=0
Sep 05 00:03:41.145326 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.145253    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27" event={"ID":"bdf63246-b183-40e3-9cc6-95ef6de59b10","Type":"ContainerDied","Data":"650074e7e9014b7da171668e959a927d18a13ebd2a81a022ef8b4cc0d4725bda"}
Sep 05 00:03:41.145326 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.145294    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27" event={"ID":"bdf63246-b183-40e3-9cc6-95ef6de59b10","Type":"ContainerDied","Data":"190fb7a247fa45c8afef5857ba5c74b961128c5100c88aedb00aac825678a3a1"}
Sep 05 00:03:41.145496 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.145475    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"
Sep 05 00:03:41.147508 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.147449    2564 generic.go:356] "Generic (PLEG): container finished" podID="760c36ca-6e89-446e-a69b-dbcc57721ba0" containerID="5c820e62659dcbbc798b76f5f591a303ce64647c0a2c7082a24d989481ef68ec" exitCode=0
Sep 05 00:03:41.148153 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.147560    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd" event={"ID":"760c36ca-6e89-446e-a69b-dbcc57721ba0","Type":"ContainerDied","Data":"5c820e62659dcbbc798b76f5f591a303ce64647c0a2c7082a24d989481ef68ec"}
Sep 05 00:03:41.148153 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.147598    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd" event={"ID":"760c36ca-6e89-446e-a69b-dbcc57721ba0","Type":"ContainerDied","Data":"31809bae956680e3fc93477e989a6f8429122c8e014275a98b98ad6b6e9b80f2"}
Sep 05 00:03:41.148153 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.147671    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"
Sep 05 00:03:41.154696 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.154676    2564 scope.go:117] "RemoveContainer" containerID="c057694e7646367515b3b42d117f2e286877d7e31c60e2c9d76dc03e6d872f10"
Sep 05 00:03:41.155084 ip-10-0-139-127 kubenswrapper[2564]: E0905 00:03:41.155058    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c057694e7646367515b3b42d117f2e286877d7e31c60e2c9d76dc03e6d872f10\": container with ID starting with c057694e7646367515b3b42d117f2e286877d7e31c60e2c9d76dc03e6d872f10 not found: ID does not exist" containerID="c057694e7646367515b3b42d117f2e286877d7e31c60e2c9d76dc03e6d872f10"
Sep 05 00:03:41.155200 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.155096    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c057694e7646367515b3b42d117f2e286877d7e31c60e2c9d76dc03e6d872f10"} err="failed to get container status \"c057694e7646367515b3b42d117f2e286877d7e31c60e2c9d76dc03e6d872f10\": rpc error: code = NotFound desc = could not find container \"c057694e7646367515b3b42d117f2e286877d7e31c60e2c9d76dc03e6d872f10\": container with ID starting with c057694e7646367515b3b42d117f2e286877d7e31c60e2c9d76dc03e6d872f10 not found: ID does not exist"
Sep 05 00:03:41.155200 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.155118    2564 scope.go:117] "RemoveContainer" containerID="650074e7e9014b7da171668e959a927d18a13ebd2a81a022ef8b4cc0d4725bda"
Sep 05 00:03:41.164167 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.164144    2564 scope.go:117] "RemoveContainer" containerID="650074e7e9014b7da171668e959a927d18a13ebd2a81a022ef8b4cc0d4725bda"
Sep 05 00:03:41.164593 ip-10-0-139-127 kubenswrapper[2564]: E0905 00:03:41.164474    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"650074e7e9014b7da171668e959a927d18a13ebd2a81a022ef8b4cc0d4725bda\": container with ID starting with 650074e7e9014b7da171668e959a927d18a13ebd2a81a022ef8b4cc0d4725bda not found: ID does not exist" containerID="650074e7e9014b7da171668e959a927d18a13ebd2a81a022ef8b4cc0d4725bda"
Sep 05 00:03:41.164593 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.164505    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"650074e7e9014b7da171668e959a927d18a13ebd2a81a022ef8b4cc0d4725bda"} err="failed to get container status \"650074e7e9014b7da171668e959a927d18a13ebd2a81a022ef8b4cc0d4725bda\": rpc error: code = NotFound desc = could not find container \"650074e7e9014b7da171668e959a927d18a13ebd2a81a022ef8b4cc0d4725bda\": container with ID starting with 650074e7e9014b7da171668e959a927d18a13ebd2a81a022ef8b4cc0d4725bda not found: ID does not exist"
Sep 05 00:03:41.164593 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.164521    2564 scope.go:117] "RemoveContainer" containerID="5c820e62659dcbbc798b76f5f591a303ce64647c0a2c7082a24d989481ef68ec"
Sep 05 00:03:41.172478 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.172454    2564 scope.go:117] "RemoveContainer" containerID="5c820e62659dcbbc798b76f5f591a303ce64647c0a2c7082a24d989481ef68ec"
Sep 05 00:03:41.172718 ip-10-0-139-127 kubenswrapper[2564]: E0905 00:03:41.172704    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c820e62659dcbbc798b76f5f591a303ce64647c0a2c7082a24d989481ef68ec\": container with ID starting with 5c820e62659dcbbc798b76f5f591a303ce64647c0a2c7082a24d989481ef68ec not found: ID does not exist" containerID="5c820e62659dcbbc798b76f5f591a303ce64647c0a2c7082a24d989481ef68ec"
Sep 05 00:03:41.172771 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.172724    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c820e62659dcbbc798b76f5f591a303ce64647c0a2c7082a24d989481ef68ec"} err="failed to get container status \"5c820e62659dcbbc798b76f5f591a303ce64647c0a2c7082a24d989481ef68ec\": rpc error: code = NotFound desc = could not find container \"5c820e62659dcbbc798b76f5f591a303ce64647c0a2c7082a24d989481ef68ec\": container with ID starting with 5c820e62659dcbbc798b76f5f591a303ce64647c0a2c7082a24d989481ef68ec not found: ID does not exist"
Sep 05 00:03:41.206445 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.206426    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-openshift-service-ca\") pod \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") "
Sep 05 00:03:41.206527 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.206494    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lx4q\" (UniqueName: \"kubernetes.io/projected/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-kube-api-access-9lx4q\") pod \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") "
Sep 05 00:03:41.206568 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.206534    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-metrics-certs\") pod \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") "
Sep 05 00:03:41.206630 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.206607    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/760c36ca-6e89-446e-a69b-dbcc57721ba0-plugins-config-volume\") pod \"760c36ca-6e89-446e-a69b-dbcc57721ba0\" (UID: \"760c36ca-6e89-446e-a69b-dbcc57721ba0\") "
Sep 05 00:03:41.206696 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.206650    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-maas-api-tls\") pod \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\" (UID: \"64851d1b-dc9a-4d7a-8ba2-a803396da2fe\") "
Sep 05 00:03:41.206754 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.206699    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fh2wk\" (UniqueName: \"kubernetes.io/projected/760c36ca-6e89-446e-a69b-dbcc57721ba0-kube-api-access-fh2wk\") pod \"760c36ca-6e89-446e-a69b-dbcc57721ba0\" (UID: \"760c36ca-6e89-446e-a69b-dbcc57721ba0\") "
Sep 05 00:03:41.206815 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.206768    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "64851d1b-dc9a-4d7a-8ba2-a803396da2fe" (UID: "64851d1b-dc9a-4d7a-8ba2-a803396da2fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 00:03:41.207088 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.207052    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/760c36ca-6e89-446e-a69b-dbcc57721ba0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "760c36ca-6e89-446e-a69b-dbcc57721ba0" (UID: "760c36ca-6e89-446e-a69b-dbcc57721ba0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 00:03:41.207088 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.207069    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:03:41.208697 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.208675    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-kube-api-access-9lx4q" (OuterVolumeSpecName: "kube-api-access-9lx4q") pod "64851d1b-dc9a-4d7a-8ba2-a803396da2fe" (UID: "64851d1b-dc9a-4d7a-8ba2-a803396da2fe"). InnerVolumeSpecName "kube-api-access-9lx4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 00:03:41.208764 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.208694    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "64851d1b-dc9a-4d7a-8ba2-a803396da2fe" (UID: "64851d1b-dc9a-4d7a-8ba2-a803396da2fe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 00:03:41.209399 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.209377    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/760c36ca-6e89-446e-a69b-dbcc57721ba0-kube-api-access-fh2wk" (OuterVolumeSpecName: "kube-api-access-fh2wk") pod "760c36ca-6e89-446e-a69b-dbcc57721ba0" (UID: "760c36ca-6e89-446e-a69b-dbcc57721ba0"). InnerVolumeSpecName "kube-api-access-fh2wk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 00:03:41.209399 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.209383    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "64851d1b-dc9a-4d7a-8ba2-a803396da2fe" (UID: "64851d1b-dc9a-4d7a-8ba2-a803396da2fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 00:03:41.307454 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.307405    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdf63246-b183-40e3-9cc6-95ef6de59b10-plugins-config-volume\") pod \"bdf63246-b183-40e3-9cc6-95ef6de59b10\" (UID: \"bdf63246-b183-40e3-9cc6-95ef6de59b10\") "
Sep 05 00:03:41.307454 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.307439    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftzlk\" (UniqueName: \"kubernetes.io/projected/bdf63246-b183-40e3-9cc6-95ef6de59b10-kube-api-access-ftzlk\") pod \"bdf63246-b183-40e3-9cc6-95ef6de59b10\" (UID: \"bdf63246-b183-40e3-9cc6-95ef6de59b10\") "
Sep 05 00:03:41.307646 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.307625    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lx4q\" (UniqueName: \"kubernetes.io/projected/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-kube-api-access-9lx4q\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:03:41.307646 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.307647    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-metrics-certs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:03:41.307778 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.307656    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/760c36ca-6e89-446e-a69b-dbcc57721ba0-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:03:41.307778 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.307665    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64851d1b-dc9a-4d7a-8ba2-a803396da2fe-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:03:41.307778 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.307673    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fh2wk\" (UniqueName: \"kubernetes.io/projected/760c36ca-6e89-446e-a69b-dbcc57721ba0-kube-api-access-fh2wk\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:03:41.307778 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.307735    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdf63246-b183-40e3-9cc6-95ef6de59b10-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bdf63246-b183-40e3-9cc6-95ef6de59b10" (UID: "bdf63246-b183-40e3-9cc6-95ef6de59b10"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 00:03:41.309318 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.309299    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdf63246-b183-40e3-9cc6-95ef6de59b10-kube-api-access-ftzlk" (OuterVolumeSpecName: "kube-api-access-ftzlk") pod "bdf63246-b183-40e3-9cc6-95ef6de59b10" (UID: "bdf63246-b183-40e3-9cc6-95ef6de59b10"). InnerVolumeSpecName "kube-api-access-ftzlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 00:03:41.408869 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.408843    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdf63246-b183-40e3-9cc6-95ef6de59b10-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:03:41.408869 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.408865    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftzlk\" (UniqueName: \"kubernetes.io/projected/bdf63246-b183-40e3-9cc6-95ef6de59b10-kube-api-access-ftzlk\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:03:41.468555 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.468533    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"]
Sep 05 00:03:41.473831 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.473811    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-64702c-6c55c6c894-qlk27"]
Sep 05 00:03:41.486307 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.486286    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"]
Sep 05 00:03:41.491621 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.491599    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-64702c-78fcb4fc5b-j5std"]
Sep 05 00:03:41.501949 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.501925    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"]
Sep 05 00:03:41.503878 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:41.503858    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-64702c-7d8c8f8d87-rpmpd"]
Sep 05 00:03:42.174673 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:42.174640    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64851d1b-dc9a-4d7a-8ba2-a803396da2fe" path="/var/lib/kubelet/pods/64851d1b-dc9a-4d7a-8ba2-a803396da2fe/volumes"
Sep 05 00:03:42.175160 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:42.175140    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="760c36ca-6e89-446e-a69b-dbcc57721ba0" path="/var/lib/kubelet/pods/760c36ca-6e89-446e-a69b-dbcc57721ba0/volumes"
Sep 05 00:03:42.175593 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:42.175578    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdf63246-b183-40e3-9cc6-95ef6de59b10" path="/var/lib/kubelet/pods/bdf63246-b183-40e3-9cc6-95ef6de59b10/volumes"
Sep 05 00:03:42.784093 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:42.784059    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:42.883630 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:42.883602    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:42.980951 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:42.980868    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:46.932945 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:46.932845    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:46.993616 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:46.993584    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:47.203992 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.203840    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:47.570583 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.570504    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll"]
Sep 05 00:03:47.570924 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.570886    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdf63246-b183-40e3-9cc6-95ef6de59b10" containerName="payload-pre-processing"
Sep 05 00:03:47.570924 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.570911    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdf63246-b183-40e3-9cc6-95ef6de59b10" containerName="payload-pre-processing"
Sep 05 00:03:47.570924 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.570921    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a578293c-603e-4465-8893-8802bf70f816" containerName="maas-api"
Sep 05 00:03:47.571044 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.570926    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a578293c-603e-4465-8893-8802bf70f816" containerName="maas-api"
Sep 05 00:03:47.571044 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.570951    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="760c36ca-6e89-446e-a69b-dbcc57721ba0" containerName="payload-processing"
Sep 05 00:03:47.571044 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.570958    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="760c36ca-6e89-446e-a69b-dbcc57721ba0" containerName="payload-processing"
Sep 05 00:03:47.571044 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.570973    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bafd86d2-353d-419c-8bee-522c47912a4e" containerName="payload-pre-processing"
Sep 05 00:03:47.571044 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.570978    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bafd86d2-353d-419c-8bee-522c47912a4e" containerName="payload-pre-processing"
Sep 05 00:03:47.571044 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.570987    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64851d1b-dc9a-4d7a-8ba2-a803396da2fe" containerName="maas-api"
Sep 05 00:03:47.571044 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.570992    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="64851d1b-dc9a-4d7a-8ba2-a803396da2fe" containerName="maas-api"
Sep 05 00:03:47.571246 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.571054    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a578293c-603e-4465-8893-8802bf70f816" containerName="maas-api"
Sep 05 00:03:47.571246 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.571064    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdf63246-b183-40e3-9cc6-95ef6de59b10" containerName="payload-pre-processing"
Sep 05 00:03:47.571246 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.571070    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="760c36ca-6e89-446e-a69b-dbcc57721ba0" containerName="payload-processing"
Sep 05 00:03:47.571246 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.571077    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="64851d1b-dc9a-4d7a-8ba2-a803396da2fe" containerName="maas-api"
Sep 05 00:03:47.571246 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.571081    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bafd86d2-353d-419c-8bee-522c47912a4e" containerName="payload-pre-processing"
Sep 05 00:03:47.575548 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.575531    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll"
Sep 05 00:03:47.578256 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.578235    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 00:03:47.578373 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.578240    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k4f64\""
Sep 05 00:03:47.583303 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.583280    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll"]
Sep 05 00:03:47.756653 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.756627    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/999967f9-5f3c-47e6-a3f3-ee0c82683c07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll\" (UID: \"999967f9-5f3c-47e6-a3f3-ee0c82683c07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll"
Sep 05 00:03:47.857537 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.857458    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/999967f9-5f3c-47e6-a3f3-ee0c82683c07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll\" (UID: \"999967f9-5f3c-47e6-a3f3-ee0c82683c07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll"
Sep 05 00:03:47.858056 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.858038    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/999967f9-5f3c-47e6-a3f3-ee0c82683c07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll\" (UID: \"999967f9-5f3c-47e6-a3f3-ee0c82683c07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll"
Sep 05 00:03:47.886980 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:47.886948    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll"
Sep 05 00:03:48.157583 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:48.157412    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll"]
Sep 05 00:03:48.159944 ip-10-0-139-127 kubenswrapper[2564]: W0905 00:03:48.159917    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod999967f9_5f3c_47e6_a3f3_ee0c82683c07.slice/crio-8781b090b74801336fe17b0b89703b37a01dcda9b55ed81154c8810528165725 WatchSource:0}: Error finding container 8781b090b74801336fe17b0b89703b37a01dcda9b55ed81154c8810528165725: Status 404 returned error can't find the container with id 8781b090b74801336fe17b0b89703b37a01dcda9b55ed81154c8810528165725
Sep 05 00:03:48.190851 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:48.190822    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll" event={"ID":"999967f9-5f3c-47e6-a3f3-ee0c82683c07","Type":"ContainerStarted","Data":"8781b090b74801336fe17b0b89703b37a01dcda9b55ed81154c8810528165725"}
Sep 05 00:03:49.197197 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:49.197136    2564 generic.go:356] "Generic (PLEG): container finished" podID="999967f9-5f3c-47e6-a3f3-ee0c82683c07" containerID="bc82b6e33573d79c2a6456f1af905b3fca2b9a0ada71abd52f84e2a7e38352f1" exitCode=0
Sep 05 00:03:49.197520 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:49.197237    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll" event={"ID":"999967f9-5f3c-47e6-a3f3-ee0c82683c07","Type":"ContainerDied","Data":"bc82b6e33573d79c2a6456f1af905b3fca2b9a0ada71abd52f84e2a7e38352f1"}
Sep 05 00:03:50.327948 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:50.327924    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll"
Sep 05 00:03:50.481194 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:50.481124    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/999967f9-5f3c-47e6-a3f3-ee0c82683c07-maas-api-service-ca\") pod \"999967f9-5f3c-47e6-a3f3-ee0c82683c07\" (UID: \"999967f9-5f3c-47e6-a3f3-ee0c82683c07\") "
Sep 05 00:03:50.481456 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:50.481432    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/999967f9-5f3c-47e6-a3f3-ee0c82683c07-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "999967f9-5f3c-47e6-a3f3-ee0c82683c07" (UID: "999967f9-5f3c-47e6-a3f3-ee0c82683c07"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 00:03:50.582356 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:50.582325    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/999967f9-5f3c-47e6-a3f3-ee0c82683c07-maas-api-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:03:51.208957 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:51.208925    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll" event={"ID":"999967f9-5f3c-47e6-a3f3-ee0c82683c07","Type":"ContainerDied","Data":"8781b090b74801336fe17b0b89703b37a01dcda9b55ed81154c8810528165725"}
Sep 05 00:03:51.208957 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:51.208945    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll"
Sep 05 00:03:51.209160 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:51.208974    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8781b090b74801336fe17b0b89703b37a01dcda9b55ed81154c8810528165725"
Sep 05 00:03:57.979580 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:57.979542    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:57.980463 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:57.980443    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd"]
Sep 05 00:03:57.984632 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:57.984606    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-5245c1-29809440-qppqd"]
Sep 05 00:03:58.008009 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.007983    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"]
Sep 05 00:03:58.008233 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.008206    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs" podUID="a2c29a92-d22c-468a-aa6f-e312593f6ccd" containerName="payload-processing" containerID="cri-o://246fbade8a6a33f099e45b39cbe28af9e2447283851ee3e55250077ea1c43ac7" gracePeriod=30
Sep 05 00:03:58.077438 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.077400    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:58.159633 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.159597    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll"]
Sep 05 00:03:58.162508 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.162466    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5245c1-sm9ll"]
Sep 05 00:03:58.162741 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.162711    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:58.175069 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.175038    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="999967f9-5f3c-47e6-a3f3-ee0c82683c07" path="/var/lib/kubelet/pods/999967f9-5f3c-47e6-a3f3-ee0c82683c07/volumes"
Sep 05 00:03:58.175359 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.175340    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f570bdc0-e8e4-4ecc-9cd9-fef771da8e9f" path="/var/lib/kubelet/pods/f570bdc0-e8e4-4ecc-9cd9-fef771da8e9f/volumes"
Sep 05 00:03:58.220450 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.220417    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:58.242698 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.242625    2564 generic.go:356] "Generic (PLEG): container finished" podID="a2c29a92-d22c-468a-aa6f-e312593f6ccd" containerID="246fbade8a6a33f099e45b39cbe28af9e2447283851ee3e55250077ea1c43ac7" exitCode=0
Sep 05 00:03:58.242829 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.242695    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs" event={"ID":"a2c29a92-d22c-468a-aa6f-e312593f6ccd","Type":"ContainerDied","Data":"246fbade8a6a33f099e45b39cbe28af9e2447283851ee3e55250077ea1c43ac7"}
Sep 05 00:03:58.270237 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.270215    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"
Sep 05 00:03:58.341162 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.341130    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n7zw\" (UniqueName: \"kubernetes.io/projected/a2c29a92-d22c-468a-aa6f-e312593f6ccd-kube-api-access-8n7zw\") pod \"a2c29a92-d22c-468a-aa6f-e312593f6ccd\" (UID: \"a2c29a92-d22c-468a-aa6f-e312593f6ccd\") "
Sep 05 00:03:58.341336 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.341192    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c29a92-d22c-468a-aa6f-e312593f6ccd-plugins-config-volume\") pod \"a2c29a92-d22c-468a-aa6f-e312593f6ccd\" (UID: \"a2c29a92-d22c-468a-aa6f-e312593f6ccd\") "
Sep 05 00:03:58.341591 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.341565    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2c29a92-d22c-468a-aa6f-e312593f6ccd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a2c29a92-d22c-468a-aa6f-e312593f6ccd" (UID: "a2c29a92-d22c-468a-aa6f-e312593f6ccd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 00:03:58.343465 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.343441    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c29a92-d22c-468a-aa6f-e312593f6ccd-kube-api-access-8n7zw" (OuterVolumeSpecName: "kube-api-access-8n7zw") pod "a2c29a92-d22c-468a-aa6f-e312593f6ccd" (UID: "a2c29a92-d22c-468a-aa6f-e312593f6ccd"). InnerVolumeSpecName "kube-api-access-8n7zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 00:03:58.442080 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.442047    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n7zw\" (UniqueName: \"kubernetes.io/projected/a2c29a92-d22c-468a-aa6f-e312593f6ccd-kube-api-access-8n7zw\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:03:58.442080 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.442075    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c29a92-d22c-468a-aa6f-e312593f6ccd-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:03:58.524515 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:58.524430    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:03:59.254964 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:59.254932    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"
Sep 05 00:03:59.254964 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:59.254940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs" event={"ID":"a2c29a92-d22c-468a-aa6f-e312593f6ccd","Type":"ContainerDied","Data":"4908466ce2ff8593aac7ed3af2b830255475e1575e2941045087ddf85b9c4756"}
Sep 05 00:03:59.255496 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:59.255018    2564 scope.go:117] "RemoveContainer" containerID="246fbade8a6a33f099e45b39cbe28af9e2447283851ee3e55250077ea1c43ac7"
Sep 05 00:03:59.276842 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:59.276793    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"]
Sep 05 00:03:59.278938 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:03:59.278914    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-5245c1-67f8cdf75-ln4vs"]
Sep 05 00:04:00.176754 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:00.176713    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2c29a92-d22c-468a-aa6f-e312593f6ccd" path="/var/lib/kubelet/pods/a2c29a92-d22c-468a-aa6f-e312593f6ccd/volumes"
Sep 05 00:04:04.247826 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:04.247796    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:04:04.347752 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:04.347703    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:04:04.555224 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:04.555135    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:04:05.007518 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:05.007489    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:04:15.461872 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:15.461831    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:04:15.500409 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:15.500379    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"]
Sep 05 00:04:15.500646 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:15.500587    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr" podUID="2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e" containerName="payload-processing" containerID="cri-o://9fe08ffe948e53df184d876a0f3ed66101104c72d38fe353d4cc4607f9102167" gracePeriod=30
Sep 05 00:04:15.561777 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:15.561690    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:04:15.657253 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:15.657212    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:04:15.721207 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:15.721126    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:04:15.750586 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:15.750566    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"
Sep 05 00:04:15.874821 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:15.874790    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsldm\" (UniqueName: \"kubernetes.io/projected/2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e-kube-api-access-gsldm\") pod \"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e\" (UID: \"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e\") "
Sep 05 00:04:15.874981 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:15.874865    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e-plugins-config-volume\") pod \"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e\" (UID: \"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e\") "
Sep 05 00:04:15.875237 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:15.875215    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e" (UID: "2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 00:04:15.876985 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:15.876966    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e-kube-api-access-gsldm" (OuterVolumeSpecName: "kube-api-access-gsldm") pod "2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e" (UID: "2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e"). InnerVolumeSpecName "kube-api-access-gsldm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 00:04:15.976513 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:15.976442    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsldm\" (UniqueName: \"kubernetes.io/projected/2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e-kube-api-access-gsldm\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:04:15.976513 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:15.976477    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e-plugins-config-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:04:16.012438 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:16.012408    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:04:16.340504 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:16.340440    2564 generic.go:356] "Generic (PLEG): container finished" podID="2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e" containerID="9fe08ffe948e53df184d876a0f3ed66101104c72d38fe353d4cc4607f9102167" exitCode=0
Sep 05 00:04:16.340504 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:16.340483    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"
Sep 05 00:04:16.340693 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:16.340537    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr" event={"ID":"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e","Type":"ContainerDied","Data":"9fe08ffe948e53df184d876a0f3ed66101104c72d38fe353d4cc4607f9102167"}
Sep 05 00:04:16.340693 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:16.340588    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr" event={"ID":"2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e","Type":"ContainerDied","Data":"08e416f90b015d8b6b55e541fd2fa4b1b5f4f77c4ab2475c3c12cafe5e83b8b9"}
Sep 05 00:04:16.340693 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:16.340617    2564 scope.go:117] "RemoveContainer" containerID="9fe08ffe948e53df184d876a0f3ed66101104c72d38fe353d4cc4607f9102167"
Sep 05 00:04:16.349339 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:16.349324    2564 scope.go:117] "RemoveContainer" containerID="9fe08ffe948e53df184d876a0f3ed66101104c72d38fe353d4cc4607f9102167"
Sep 05 00:04:16.349621 ip-10-0-139-127 kubenswrapper[2564]: E0905 00:04:16.349600    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fe08ffe948e53df184d876a0f3ed66101104c72d38fe353d4cc4607f9102167\": container with ID starting with 9fe08ffe948e53df184d876a0f3ed66101104c72d38fe353d4cc4607f9102167 not found: ID does not exist" containerID="9fe08ffe948e53df184d876a0f3ed66101104c72d38fe353d4cc4607f9102167"
Sep 05 00:04:16.349668 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:16.349633    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fe08ffe948e53df184d876a0f3ed66101104c72d38fe353d4cc4607f9102167"} err="failed to get container status \"9fe08ffe948e53df184d876a0f3ed66101104c72d38fe353d4cc4607f9102167\": rpc error: code = NotFound desc = could not find container \"9fe08ffe948e53df184d876a0f3ed66101104c72d38fe353d4cc4607f9102167\": container with ID starting with 9fe08ffe948e53df184d876a0f3ed66101104c72d38fe353d4cc4607f9102167 not found: ID does not exist"
Sep 05 00:04:16.355400 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:16.355375    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"]
Sep 05 00:04:16.359465 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:16.359443    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-88828d-9c4b5b877-wwlzr"]
Sep 05 00:04:18.174231 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:18.174198    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e" path="/var/lib/kubelet/pods/2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e/volumes"
Sep 05 00:04:21.853407 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:21.853366    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:04:22.002882 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:22.002850    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:04:22.135274 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:04:22.135240    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:05:45.170609 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:05:45.170569    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:05:45.812512 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:05:45.812457    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:05:47.439221 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:05:47.439188    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:05:52.469212 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:05:52.469175    2564 scope.go:117] "RemoveContainer" containerID="fbfa042e9a425c43f5863e809b51c695163f574af5f1da91efa3b48299b4b90b"
Sep 05 00:05:52.477097 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:05:52.477079    2564 scope.go:117] "RemoveContainer" containerID="2159960ded29dd0b23153320dfc4b47728f15d2c9d86183af04311edc57b0445"
Sep 05 00:06:39.637701 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:06:39.637660    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:06:52.498559 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:06:52.498529    2564 scope.go:117] "RemoveContainer" containerID="9adfab0edc8fe0a76d062fbeafacda0565c6b6538c5f559449c3ee092e29710c"
Sep 05 00:06:52.506098 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:06:52.506067    2564 scope.go:117] "RemoveContainer" containerID="db6c6423ed4725f778fb761fbdb099d38066fa24e79398d4311eaa98dde7698d"
Sep 05 00:06:56.385058 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:06:56.385024    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:06:56.435704 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:06:56.435673    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:06:58.979048 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:06:58.979015    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"]
Sep 05 00:06:58.979460 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:06:58.979223    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq" podUID="042f366d-62d3-4f8d-bcbc-cb1a272a716c" containerName="manager" containerID="cri-o://f6f3685e10a59cfc73ae5914e3f09cd121ee16f06213543dd159b280fe1d5aab" gracePeriod=10
Sep 05 00:07:01.017457 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.017429    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"
Sep 05 00:07:01.127194 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.127157    2564 generic.go:356] "Generic (PLEG): container finished" podID="042f366d-62d3-4f8d-bcbc-cb1a272a716c" containerID="f6f3685e10a59cfc73ae5914e3f09cd121ee16f06213543dd159b280fe1d5aab" exitCode=0
Sep 05 00:07:01.127375 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.127211    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"
Sep 05 00:07:01.127375 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.127240    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq" event={"ID":"042f366d-62d3-4f8d-bcbc-cb1a272a716c","Type":"ContainerDied","Data":"f6f3685e10a59cfc73ae5914e3f09cd121ee16f06213543dd159b280fe1d5aab"}
Sep 05 00:07:01.127375 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.127298    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq" event={"ID":"042f366d-62d3-4f8d-bcbc-cb1a272a716c","Type":"ContainerDied","Data":"88d2e0da94215af6858425a262626ea04bf415b569c7ffa35b28370797d8756f"}
Sep 05 00:07:01.127375 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.127329    2564 scope.go:117] "RemoveContainer" containerID="f6f3685e10a59cfc73ae5914e3f09cd121ee16f06213543dd159b280fe1d5aab"
Sep 05 00:07:01.136054 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.136034    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2n4k\" (UniqueName: \"kubernetes.io/projected/042f366d-62d3-4f8d-bcbc-cb1a272a716c-kube-api-access-c2n4k\") pod \"042f366d-62d3-4f8d-bcbc-cb1a272a716c\" (UID: \"042f366d-62d3-4f8d-bcbc-cb1a272a716c\") "
Sep 05 00:07:01.136174 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.136069    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/042f366d-62d3-4f8d-bcbc-cb1a272a716c-extensions-socket-volume\") pod \"042f366d-62d3-4f8d-bcbc-cb1a272a716c\" (UID: \"042f366d-62d3-4f8d-bcbc-cb1a272a716c\") "
Sep 05 00:07:01.136508 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.136489    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042f366d-62d3-4f8d-bcbc-cb1a272a716c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "042f366d-62d3-4f8d-bcbc-cb1a272a716c" (UID: "042f366d-62d3-4f8d-bcbc-cb1a272a716c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 00:07:01.140222 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.140201    2564 scope.go:117] "RemoveContainer" containerID="f6f3685e10a59cfc73ae5914e3f09cd121ee16f06213543dd159b280fe1d5aab"
Sep 05 00:07:01.140321 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.140288    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/042f366d-62d3-4f8d-bcbc-cb1a272a716c-kube-api-access-c2n4k" (OuterVolumeSpecName: "kube-api-access-c2n4k") pod "042f366d-62d3-4f8d-bcbc-cb1a272a716c" (UID: "042f366d-62d3-4f8d-bcbc-cb1a272a716c"). InnerVolumeSpecName "kube-api-access-c2n4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 00:07:01.140825 ip-10-0-139-127 kubenswrapper[2564]: E0905 00:07:01.140785    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6f3685e10a59cfc73ae5914e3f09cd121ee16f06213543dd159b280fe1d5aab\": container with ID starting with f6f3685e10a59cfc73ae5914e3f09cd121ee16f06213543dd159b280fe1d5aab not found: ID does not exist" containerID="f6f3685e10a59cfc73ae5914e3f09cd121ee16f06213543dd159b280fe1d5aab"
Sep 05 00:07:01.140985 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.140833    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6f3685e10a59cfc73ae5914e3f09cd121ee16f06213543dd159b280fe1d5aab"} err="failed to get container status \"f6f3685e10a59cfc73ae5914e3f09cd121ee16f06213543dd159b280fe1d5aab\": rpc error: code = NotFound desc = could not find container \"f6f3685e10a59cfc73ae5914e3f09cd121ee16f06213543dd159b280fe1d5aab\": container with ID starting with f6f3685e10a59cfc73ae5914e3f09cd121ee16f06213543dd159b280fe1d5aab not found: ID does not exist"
Sep 05 00:07:01.237123 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.237098    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2n4k\" (UniqueName: \"kubernetes.io/projected/042f366d-62d3-4f8d-bcbc-cb1a272a716c-kube-api-access-c2n4k\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:07:01.237221 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.237124    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/042f366d-62d3-4f8d-bcbc-cb1a272a716c-extensions-socket-volume\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:07:01.450768 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.450739    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"]
Sep 05 00:07:01.453835 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:01.453811    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-9wgwq"]
Sep 05 00:07:02.174262 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:02.174235    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="042f366d-62d3-4f8d-bcbc-cb1a272a716c" path="/var/lib/kubelet/pods/042f366d-62d3-4f8d-bcbc-cb1a272a716c/volumes"
Sep 05 00:07:08.051054 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:08.051018    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:07:08.105134 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:08.105098    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:07:59.655176 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.655134    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c"]
Sep 05 00:07:59.655588 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.655454    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c29a92-d22c-468a-aa6f-e312593f6ccd" containerName="payload-processing"
Sep 05 00:07:59.655588 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.655465    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c29a92-d22c-468a-aa6f-e312593f6ccd" containerName="payload-processing"
Sep 05 00:07:59.655588 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.655479    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e" containerName="payload-processing"
Sep 05 00:07:59.655588 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.655484    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e" containerName="payload-processing"
Sep 05 00:07:59.655588 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.655495    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="042f366d-62d3-4f8d-bcbc-cb1a272a716c" containerName="manager"
Sep 05 00:07:59.655588 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.655501    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="042f366d-62d3-4f8d-bcbc-cb1a272a716c" containerName="manager"
Sep 05 00:07:59.655588 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.655508    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="999967f9-5f3c-47e6-a3f3-ee0c82683c07" containerName="revoke-keys"
Sep 05 00:07:59.655588 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.655513    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="999967f9-5f3c-47e6-a3f3-ee0c82683c07" containerName="revoke-keys"
Sep 05 00:07:59.655588 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.655561    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="042f366d-62d3-4f8d-bcbc-cb1a272a716c" containerName="manager"
Sep 05 00:07:59.655588 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.655568    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c29a92-d22c-468a-aa6f-e312593f6ccd" containerName="payload-processing"
Sep 05 00:07:59.655588 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.655575    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f00d1f4-5327-4a88-bcd4-0c52e8b22e7e" containerName="payload-processing"
Sep 05 00:07:59.655588 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.655581    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="999967f9-5f3c-47e6-a3f3-ee0c82683c07" containerName="revoke-keys"
Sep 05 00:07:59.658507 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.658490    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c"
Sep 05 00:07:59.661080 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.661058    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j28nh\""
Sep 05 00:07:59.673391 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.673366    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c"]
Sep 05 00:07:59.796419 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.796392    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v88q8\" (UniqueName: \"kubernetes.io/projected/8cafd31f-a78a-4218-b193-78da58d65756-kube-api-access-v88q8\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-28f8c\" (UID: \"8cafd31f-a78a-4218-b193-78da58d65756\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c"
Sep 05 00:07:59.796545 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.796427    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cafd31f-a78a-4218-b193-78da58d65756-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-28f8c\" (UID: \"8cafd31f-a78a-4218-b193-78da58d65756\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c"
Sep 05 00:07:59.897808 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.897779    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v88q8\" (UniqueName: \"kubernetes.io/projected/8cafd31f-a78a-4218-b193-78da58d65756-kube-api-access-v88q8\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-28f8c\" (UID: \"8cafd31f-a78a-4218-b193-78da58d65756\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c"
Sep 05 00:07:59.897937 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.897813    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cafd31f-a78a-4218-b193-78da58d65756-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-28f8c\" (UID: \"8cafd31f-a78a-4218-b193-78da58d65756\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c"
Sep 05 00:07:59.898162 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.898145    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cafd31f-a78a-4218-b193-78da58d65756-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-28f8c\" (UID: \"8cafd31f-a78a-4218-b193-78da58d65756\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c"
Sep 05 00:07:59.908968 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.908920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v88q8\" (UniqueName: \"kubernetes.io/projected/8cafd31f-a78a-4218-b193-78da58d65756-kube-api-access-v88q8\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-28f8c\" (UID: \"8cafd31f-a78a-4218-b193-78da58d65756\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c"
Sep 05 00:07:59.968049 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:07:59.968025    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c"
Sep 05 00:08:00.241074 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:00.241045    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c"]
Sep 05 00:08:00.243660 ip-10-0-139-127 kubenswrapper[2564]: W0905 00:08:00.243629    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cafd31f_a78a_4218_b193_78da58d65756.slice/crio-8fc190e020352caab7b0bf1d875a3f77fdf8244fea992294d16af5fb7b6a0b3d WatchSource:0}: Error finding container 8fc190e020352caab7b0bf1d875a3f77fdf8244fea992294d16af5fb7b6a0b3d: Status 404 returned error can't find the container with id 8fc190e020352caab7b0bf1d875a3f77fdf8244fea992294d16af5fb7b6a0b3d
Sep 05 00:08:00.396392 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:00.396349    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c" event={"ID":"8cafd31f-a78a-4218-b193-78da58d65756","Type":"ContainerStarted","Data":"38651bd294377da9d3689f7a3a911b7dcecf2b2a8803fc3704a7090efe647ffa"}
Sep 05 00:08:00.396579 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:00.396415    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c" event={"ID":"8cafd31f-a78a-4218-b193-78da58d65756","Type":"ContainerStarted","Data":"8fc190e020352caab7b0bf1d875a3f77fdf8244fea992294d16af5fb7b6a0b3d"}
Sep 05 00:08:00.396579 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:00.396461    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c"
Sep 05 00:08:00.418218 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:00.418167    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c" podStartSLOduration=1.418152767 podStartE2EDuration="1.418152767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 00:07:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 00:08:00.416702697 +0000 UTC m=+1868.664703217" watchObservedRunningTime="2026-09-05 00:08:00.418152767 +0000 UTC m=+1868.666153282"
Sep 05 00:08:11.403889 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:11.403856    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-28f8c"
Sep 05 00:08:49.421211 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.421176    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"]
Sep 05 00:08:49.424938 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.424915    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.427465 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.427443    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-4ef033b4-kserve-self-signed-certs\""
Sep 05 00:08:49.431654 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.431628    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"]
Sep 05 00:08:49.474196 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.474163    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:08:49.545784 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.545750    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn7c8\" (UniqueName: \"kubernetes.io/projected/a6ee2efc-833c-4490-9a59-0311107cb4d9-kube-api-access-qn7c8\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.545993 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.545792    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6ee2efc-833c-4490-9a59-0311107cb4d9-tls-certs\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.545993 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.545840    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-tmp-dir\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.545993 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.545860    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-model-cache\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.545993 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.545882    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-home\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.545993 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.545968    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-dshm\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.646429 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.646393    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn7c8\" (UniqueName: \"kubernetes.io/projected/a6ee2efc-833c-4490-9a59-0311107cb4d9-kube-api-access-qn7c8\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.646429 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.646445    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6ee2efc-833c-4490-9a59-0311107cb4d9-tls-certs\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.646714 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.646517    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-tmp-dir\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.646714 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.646547    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-model-cache\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.646714 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.646588    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-home\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.646714 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.646618    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-dshm\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.647229 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.647034    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-tmp-dir\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.647229 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.647146    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-home\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.647365 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.647256    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-model-cache\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.650046 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.650023    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-dshm\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.650351 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.650331    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6ee2efc-833c-4490-9a59-0311107cb4d9-tls-certs\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.654422 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.654396    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn7c8\" (UniqueName: \"kubernetes.io/projected/a6ee2efc-833c-4490-9a59-0311107cb4d9-kube-api-access-qn7c8\") pod \"e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") " pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:49.735272 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:49.735191    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:50.011542 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:50.010855    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"]
Sep 05 00:08:50.013539 ip-10-0-139-127 kubenswrapper[2564]: W0905 00:08:50.013509    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6ee2efc_833c_4490_9a59_0311107cb4d9.slice/crio-8c13e0e5f971147e8ddaebc31486be1a3e5fc52990249e36852a1be2002e379b WatchSource:0}: Error finding container 8c13e0e5f971147e8ddaebc31486be1a3e5fc52990249e36852a1be2002e379b: Status 404 returned error can't find the container with id 8c13e0e5f971147e8ddaebc31486be1a3e5fc52990249e36852a1be2002e379b
Sep 05 00:08:50.630954 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:50.630914    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4" event={"ID":"a6ee2efc-833c-4490-9a59-0311107cb4d9","Type":"ContainerStarted","Data":"d9e9883aa002be8c7e11124b800770b7cd63efcb23e6adc3e7614001faba8335"}
Sep 05 00:08:50.630954 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:50.630954    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4" event={"ID":"a6ee2efc-833c-4490-9a59-0311107cb4d9","Type":"ContainerStarted","Data":"8c13e0e5f971147e8ddaebc31486be1a3e5fc52990249e36852a1be2002e379b"}
Sep 05 00:08:50.646702 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:50.646652    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4" podStartSLOduration=1.64663381 podStartE2EDuration="1.64663381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 00:08:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 00:08:50.64663066 +0000 UTC m=+1918.894631211" watchObservedRunningTime="2026-09-05 00:08:50.64663381 +0000 UTC m=+1918.894634326"
Sep 05 00:08:55.242283 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:55.242254    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:08:59.735711 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:59.735677    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:59.736145 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:59.735753    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:08:59.747703 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:08:59.747682    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:09:00.694200 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:00.694171    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:09:26.065605 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:26.065567    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"]
Sep 05 00:09:26.082864 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:26.082829    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:09:26.914097 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:26.914052    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:09:41.086661 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.086622    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4" podUID="a6ee2efc-833c-4490-9a59-0311107cb4d9" containerName="main" containerID="cri-o://d9e9883aa002be8c7e11124b800770b7cd63efcb23e6adc3e7614001faba8335" gracePeriod=45
Sep 05 00:09:41.326082 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.326058    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:09:41.493489 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.493454    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-home\") pod \"a6ee2efc-833c-4490-9a59-0311107cb4d9\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") "
Sep 05 00:09:41.493489 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.493489    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-model-cache\") pod \"a6ee2efc-833c-4490-9a59-0311107cb4d9\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") "
Sep 05 00:09:41.493729 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.493548    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6ee2efc-833c-4490-9a59-0311107cb4d9-tls-certs\") pod \"a6ee2efc-833c-4490-9a59-0311107cb4d9\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") "
Sep 05 00:09:41.493729 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.493600    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qn7c8\" (UniqueName: \"kubernetes.io/projected/a6ee2efc-833c-4490-9a59-0311107cb4d9-kube-api-access-qn7c8\") pod \"a6ee2efc-833c-4490-9a59-0311107cb4d9\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") "
Sep 05 00:09:41.493729 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.493624    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-dshm\") pod \"a6ee2efc-833c-4490-9a59-0311107cb4d9\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") "
Sep 05 00:09:41.493729 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.493662    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-tmp-dir\") pod \"a6ee2efc-833c-4490-9a59-0311107cb4d9\" (UID: \"a6ee2efc-833c-4490-9a59-0311107cb4d9\") "
Sep 05 00:09:41.493939 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.493725    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-home" (OuterVolumeSpecName: "home") pod "a6ee2efc-833c-4490-9a59-0311107cb4d9" (UID: "a6ee2efc-833c-4490-9a59-0311107cb4d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 00:09:41.493939 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.493827    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "a6ee2efc-833c-4490-9a59-0311107cb4d9" (UID: "a6ee2efc-833c-4490-9a59-0311107cb4d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 00:09:41.494066 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.493962    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-home\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:09:41.494066 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.493981    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-model-cache\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:09:41.494154 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.494083    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a6ee2efc-833c-4490-9a59-0311107cb4d9" (UID: "a6ee2efc-833c-4490-9a59-0311107cb4d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 00:09:41.495966 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.495938    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-dshm" (OuterVolumeSpecName: "dshm") pod "a6ee2efc-833c-4490-9a59-0311107cb4d9" (UID: "a6ee2efc-833c-4490-9a59-0311107cb4d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 00:09:41.496099 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.496030    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6ee2efc-833c-4490-9a59-0311107cb4d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a6ee2efc-833c-4490-9a59-0311107cb4d9" (UID: "a6ee2efc-833c-4490-9a59-0311107cb4d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 00:09:41.496099 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.496055    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6ee2efc-833c-4490-9a59-0311107cb4d9-kube-api-access-qn7c8" (OuterVolumeSpecName: "kube-api-access-qn7c8") pod "a6ee2efc-833c-4490-9a59-0311107cb4d9" (UID: "a6ee2efc-833c-4490-9a59-0311107cb4d9"). InnerVolumeSpecName "kube-api-access-qn7c8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 00:09:41.594708 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.594672    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6ee2efc-833c-4490-9a59-0311107cb4d9-tls-certs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:09:41.594708 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.594702    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qn7c8\" (UniqueName: \"kubernetes.io/projected/a6ee2efc-833c-4490-9a59-0311107cb4d9-kube-api-access-qn7c8\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:09:41.594708 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.594712    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-dshm\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:09:41.594978 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.594721    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ee2efc-833c-4490-9a59-0311107cb4d9-tmp-dir\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:09:41.868634 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.868595    2564 generic.go:356] "Generic (PLEG): container finished" podID="a6ee2efc-833c-4490-9a59-0311107cb4d9" containerID="d9e9883aa002be8c7e11124b800770b7cd63efcb23e6adc3e7614001faba8335" exitCode=0
Sep 05 00:09:41.868807 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.868638    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"
Sep 05 00:09:41.868807 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.868679    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4" event={"ID":"a6ee2efc-833c-4490-9a59-0311107cb4d9","Type":"ContainerDied","Data":"d9e9883aa002be8c7e11124b800770b7cd63efcb23e6adc3e7614001faba8335"}
Sep 05 00:09:41.868807 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.868728    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4" event={"ID":"a6ee2efc-833c-4490-9a59-0311107cb4d9","Type":"ContainerDied","Data":"8c13e0e5f971147e8ddaebc31486be1a3e5fc52990249e36852a1be2002e379b"}
Sep 05 00:09:41.868807 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.868753    2564 scope.go:117] "RemoveContainer" containerID="d9e9883aa002be8c7e11124b800770b7cd63efcb23e6adc3e7614001faba8335"
Sep 05 00:09:41.878084 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.878062    2564 scope.go:117] "RemoveContainer" containerID="d9e9883aa002be8c7e11124b800770b7cd63efcb23e6adc3e7614001faba8335"
Sep 05 00:09:41.878374 ip-10-0-139-127 kubenswrapper[2564]: E0905 00:09:41.878356    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9e9883aa002be8c7e11124b800770b7cd63efcb23e6adc3e7614001faba8335\": container with ID starting with d9e9883aa002be8c7e11124b800770b7cd63efcb23e6adc3e7614001faba8335 not found: ID does not exist" containerID="d9e9883aa002be8c7e11124b800770b7cd63efcb23e6adc3e7614001faba8335"
Sep 05 00:09:41.878451 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.878382    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e9883aa002be8c7e11124b800770b7cd63efcb23e6adc3e7614001faba8335"} err="failed to get container status \"d9e9883aa002be8c7e11124b800770b7cd63efcb23e6adc3e7614001faba8335\": rpc error: code = NotFound desc = could not find container \"d9e9883aa002be8c7e11124b800770b7cd63efcb23e6adc3e7614001faba8335\": container with ID starting with d9e9883aa002be8c7e11124b800770b7cd63efcb23e6adc3e7614001faba8335 not found: ID does not exist"
Sep 05 00:09:41.890294 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.890267    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"]
Sep 05 00:09:41.891824 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:41.891799    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-4ef033b4-kserve-7d6757bff5-wvnl4"]
Sep 05 00:09:42.175682 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:42.175590    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6ee2efc-833c-4490-9a59-0311107cb4d9" path="/var/lib/kubelet/pods/a6ee2efc-833c-4490-9a59-0311107cb4d9/volumes"
Sep 05 00:09:43.866515 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:43.866430    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"]
Sep 05 00:09:43.866873 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:43.866771    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ee2efc-833c-4490-9a59-0311107cb4d9" containerName="main"
Sep 05 00:09:43.866873 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:43.866782    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ee2efc-833c-4490-9a59-0311107cb4d9" containerName="main"
Sep 05 00:09:43.866873 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:43.866863    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6ee2efc-833c-4490-9a59-0311107cb4d9" containerName="main"
Sep 05 00:09:43.871193 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:43.871172    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:43.875473 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:43.875446    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 05 00:09:43.875617 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:43.875601    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 00:09:43.875697 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:43.875677    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 05 00:09:43.875761 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:43.875603    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mnd5p\""
Sep 05 00:09:43.876010 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:43.875996    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 05 00:09:43.880137 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:43.880109    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"]
Sep 05 00:09:44.017208 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.017167    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a30b88-b353-4314-8445-e8f7ed7635c2-metrics-certs\") pod \"maas-api-66b67c778f-rx9hx\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") " pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:44.017208 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.017213    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t9cx\" (UniqueName: \"kubernetes.io/projected/57a30b88-b353-4314-8445-e8f7ed7635c2-kube-api-access-8t9cx\") pod \"maas-api-66b67c778f-rx9hx\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") " pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:44.017431 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.017278    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57a30b88-b353-4314-8445-e8f7ed7635c2-openshift-service-ca\") pod \"maas-api-66b67c778f-rx9hx\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") " pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:44.017431 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.017296    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57a30b88-b353-4314-8445-e8f7ed7635c2-maas-api-tls\") pod \"maas-api-66b67c778f-rx9hx\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") " pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:44.118372 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.118273    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57a30b88-b353-4314-8445-e8f7ed7635c2-openshift-service-ca\") pod \"maas-api-66b67c778f-rx9hx\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") " pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:44.118372 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.118316    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57a30b88-b353-4314-8445-e8f7ed7635c2-maas-api-tls\") pod \"maas-api-66b67c778f-rx9hx\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") " pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:44.118372 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.118370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a30b88-b353-4314-8445-e8f7ed7635c2-metrics-certs\") pod \"maas-api-66b67c778f-rx9hx\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") " pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:44.118660 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.118388    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9cx\" (UniqueName: \"kubernetes.io/projected/57a30b88-b353-4314-8445-e8f7ed7635c2-kube-api-access-8t9cx\") pod \"maas-api-66b67c778f-rx9hx\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") " pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:44.119017 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.118997    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57a30b88-b353-4314-8445-e8f7ed7635c2-openshift-service-ca\") pod \"maas-api-66b67c778f-rx9hx\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") " pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:44.120926 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.120886    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a30b88-b353-4314-8445-e8f7ed7635c2-metrics-certs\") pod \"maas-api-66b67c778f-rx9hx\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") " pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:44.121013 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.120946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57a30b88-b353-4314-8445-e8f7ed7635c2-maas-api-tls\") pod \"maas-api-66b67c778f-rx9hx\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") " pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:44.126201 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.126171    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t9cx\" (UniqueName: \"kubernetes.io/projected/57a30b88-b353-4314-8445-e8f7ed7635c2-kube-api-access-8t9cx\") pod \"maas-api-66b67c778f-rx9hx\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") " pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:44.183319 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.183286    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:44.454838 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.454807    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"]
Sep 05 00:09:44.456795 ip-10-0-139-127 kubenswrapper[2564]: W0905 00:09:44.456768    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57a30b88_b353_4314_8445_e8f7ed7635c2.slice/crio-04427d3bd441074ead5595d4846eee5b020b2aa44d29eb24e45b50081880fa07 WatchSource:0}: Error finding container 04427d3bd441074ead5595d4846eee5b020b2aa44d29eb24e45b50081880fa07: Status 404 returned error can't find the container with id 04427d3bd441074ead5595d4846eee5b020b2aa44d29eb24e45b50081880fa07
Sep 05 00:09:44.458415 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.458394    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 00:09:44.886819 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:44.886783    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx" event={"ID":"57a30b88-b353-4314-8445-e8f7ed7635c2","Type":"ContainerStarted","Data":"04427d3bd441074ead5595d4846eee5b020b2aa44d29eb24e45b50081880fa07"}
Sep 05 00:09:45.893128 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:45.893094    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx" event={"ID":"57a30b88-b353-4314-8445-e8f7ed7635c2","Type":"ContainerStarted","Data":"f417afe0ee0c86127838c1ec84807cdc913ad70aab651e259e537ef923bb27fa"}
Sep 05 00:09:45.893545 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:45.893159    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:45.909806 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:45.909761    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx" podStartSLOduration=2.223899454 podStartE2EDuration="2.90974701s" totalImagesPullingTime="685.847556ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 00:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 00:09:45.908810149 +0000 UTC m=+1974.156810665" watchObservedRunningTime="2026-09-05 00:09:45.90974701 +0000 UTC m=+1974.157747554"
Sep 05 00:09:51.904211 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:51.904179    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:09:52.046936 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.046889    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-57989d57b9-shp75"]
Sep 05 00:09:52.050654 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.050633    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.059343 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.059320    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57989d57b9-shp75"]
Sep 05 00:09:52.190371 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.190305    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df1bfa98-c2d8-4408-8458-9a2c106dc02c-openshift-service-ca\") pod \"maas-api-57989d57b9-shp75\" (UID: \"df1bfa98-c2d8-4408-8458-9a2c106dc02c\") " pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.190371 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.190343    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5wgg\" (UniqueName: \"kubernetes.io/projected/df1bfa98-c2d8-4408-8458-9a2c106dc02c-kube-api-access-f5wgg\") pod \"maas-api-57989d57b9-shp75\" (UID: \"df1bfa98-c2d8-4408-8458-9a2c106dc02c\") " pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.190525 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.190368    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df1bfa98-c2d8-4408-8458-9a2c106dc02c-maas-api-tls\") pod \"maas-api-57989d57b9-shp75\" (UID: \"df1bfa98-c2d8-4408-8458-9a2c106dc02c\") " pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.190525 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.190466    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1bfa98-c2d8-4408-8458-9a2c106dc02c-metrics-certs\") pod \"maas-api-57989d57b9-shp75\" (UID: \"df1bfa98-c2d8-4408-8458-9a2c106dc02c\") " pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.291245 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.291221    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1bfa98-c2d8-4408-8458-9a2c106dc02c-metrics-certs\") pod \"maas-api-57989d57b9-shp75\" (UID: \"df1bfa98-c2d8-4408-8458-9a2c106dc02c\") " pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.291373 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.291261    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df1bfa98-c2d8-4408-8458-9a2c106dc02c-openshift-service-ca\") pod \"maas-api-57989d57b9-shp75\" (UID: \"df1bfa98-c2d8-4408-8458-9a2c106dc02c\") " pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.291373 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.291283    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5wgg\" (UniqueName: \"kubernetes.io/projected/df1bfa98-c2d8-4408-8458-9a2c106dc02c-kube-api-access-f5wgg\") pod \"maas-api-57989d57b9-shp75\" (UID: \"df1bfa98-c2d8-4408-8458-9a2c106dc02c\") " pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.291373 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.291308    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df1bfa98-c2d8-4408-8458-9a2c106dc02c-maas-api-tls\") pod \"maas-api-57989d57b9-shp75\" (UID: \"df1bfa98-c2d8-4408-8458-9a2c106dc02c\") " pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.291846 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.291781    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df1bfa98-c2d8-4408-8458-9a2c106dc02c-openshift-service-ca\") pod \"maas-api-57989d57b9-shp75\" (UID: \"df1bfa98-c2d8-4408-8458-9a2c106dc02c\") " pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.293573 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.293546    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df1bfa98-c2d8-4408-8458-9a2c106dc02c-metrics-certs\") pod \"maas-api-57989d57b9-shp75\" (UID: \"df1bfa98-c2d8-4408-8458-9a2c106dc02c\") " pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.293688 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.293675    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df1bfa98-c2d8-4408-8458-9a2c106dc02c-maas-api-tls\") pod \"maas-api-57989d57b9-shp75\" (UID: \"df1bfa98-c2d8-4408-8458-9a2c106dc02c\") " pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.303725 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.303702    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5wgg\" (UniqueName: \"kubernetes.io/projected/df1bfa98-c2d8-4408-8458-9a2c106dc02c-kube-api-access-f5wgg\") pod \"maas-api-57989d57b9-shp75\" (UID: \"df1bfa98-c2d8-4408-8458-9a2c106dc02c\") " pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.360757 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.360737    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:52.547061 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.546983    2564 scope.go:117] "RemoveContainer" containerID="bc82b6e33573d79c2a6456f1af905b3fca2b9a0ada71abd52f84e2a7e38352f1"
Sep 05 00:09:52.632853 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.632812    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57989d57b9-shp75"]
Sep 05 00:09:52.634295 ip-10-0-139-127 kubenswrapper[2564]: W0905 00:09:52.634272    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf1bfa98_c2d8_4408_8458_9a2c106dc02c.slice/crio-bd0f657f243e7144858e0bd33581d8ab5e162008125e694af238f6efd6a5df36 WatchSource:0}: Error finding container bd0f657f243e7144858e0bd33581d8ab5e162008125e694af238f6efd6a5df36: Status 404 returned error can't find the container with id bd0f657f243e7144858e0bd33581d8ab5e162008125e694af238f6efd6a5df36
Sep 05 00:09:52.929021 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:52.928987    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75" event={"ID":"df1bfa98-c2d8-4408-8458-9a2c106dc02c","Type":"ContainerStarted","Data":"bd0f657f243e7144858e0bd33581d8ab5e162008125e694af238f6efd6a5df36"}
Sep 05 00:09:53.934877 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:53.934843    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75" event={"ID":"df1bfa98-c2d8-4408-8458-9a2c106dc02c","Type":"ContainerStarted","Data":"dd96b95e8374cec49a3e24ec4e00f907d69a664bde4538caaca62def54d87807"}
Sep 05 00:09:53.935267 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:53.934957    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:53.951884 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:53.951841    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75" podStartSLOduration=1.438185069 podStartE2EDuration="1.951828393s" totalImagesPullingTime="513.643324ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 00:09:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 00:09:53.951629864 +0000 UTC m=+1982.199630396" watchObservedRunningTime="2026-09-05 00:09:53.951828393 +0000 UTC m=+1982.199828908"
Sep 05 00:09:59.946423 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:59.946395    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-57989d57b9-shp75"
Sep 05 00:09:59.989350 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:59.989320    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"]
Sep 05 00:09:59.989531 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:09:59.989507    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx" podUID="57a30b88-b353-4314-8445-e8f7ed7635c2" containerName="maas-api" containerID="cri-o://f417afe0ee0c86127838c1ec84807cdc913ad70aab651e259e537ef923bb27fa" gracePeriod=30
Sep 05 00:10:01.223757 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.223731    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:10:01.270177 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.270148    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t9cx\" (UniqueName: \"kubernetes.io/projected/57a30b88-b353-4314-8445-e8f7ed7635c2-kube-api-access-8t9cx\") pod \"57a30b88-b353-4314-8445-e8f7ed7635c2\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") "
Sep 05 00:10:01.270304 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.270201    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57a30b88-b353-4314-8445-e8f7ed7635c2-openshift-service-ca\") pod \"57a30b88-b353-4314-8445-e8f7ed7635c2\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") "
Sep 05 00:10:01.270304 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.270232    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a30b88-b353-4314-8445-e8f7ed7635c2-metrics-certs\") pod \"57a30b88-b353-4314-8445-e8f7ed7635c2\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") "
Sep 05 00:10:01.270383 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.270331    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57a30b88-b353-4314-8445-e8f7ed7635c2-maas-api-tls\") pod \"57a30b88-b353-4314-8445-e8f7ed7635c2\" (UID: \"57a30b88-b353-4314-8445-e8f7ed7635c2\") "
Sep 05 00:10:01.270783 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.270602    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57a30b88-b353-4314-8445-e8f7ed7635c2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "57a30b88-b353-4314-8445-e8f7ed7635c2" (UID: "57a30b88-b353-4314-8445-e8f7ed7635c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 00:10:01.272505 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.272473    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57a30b88-b353-4314-8445-e8f7ed7635c2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "57a30b88-b353-4314-8445-e8f7ed7635c2" (UID: "57a30b88-b353-4314-8445-e8f7ed7635c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 00:10:01.272505 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.272476    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57a30b88-b353-4314-8445-e8f7ed7635c2-kube-api-access-8t9cx" (OuterVolumeSpecName: "kube-api-access-8t9cx") pod "57a30b88-b353-4314-8445-e8f7ed7635c2" (UID: "57a30b88-b353-4314-8445-e8f7ed7635c2"). InnerVolumeSpecName "kube-api-access-8t9cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 00:10:01.272634 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.272561    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57a30b88-b353-4314-8445-e8f7ed7635c2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "57a30b88-b353-4314-8445-e8f7ed7635c2" (UID: "57a30b88-b353-4314-8445-e8f7ed7635c2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 00:10:01.371296 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.371275    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t9cx\" (UniqueName: \"kubernetes.io/projected/57a30b88-b353-4314-8445-e8f7ed7635c2-kube-api-access-8t9cx\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:10:01.371296 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.371295    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57a30b88-b353-4314-8445-e8f7ed7635c2-openshift-service-ca\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:10:01.371412 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.371305    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57a30b88-b353-4314-8445-e8f7ed7635c2-metrics-certs\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:10:01.371412 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.371313    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57a30b88-b353-4314-8445-e8f7ed7635c2-maas-api-tls\") on node \"ip-10-0-139-127.ec2.internal\" DevicePath \"\""
Sep 05 00:10:01.975218 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.975181    2564 generic.go:356] "Generic (PLEG): container finished" podID="57a30b88-b353-4314-8445-e8f7ed7635c2" containerID="f417afe0ee0c86127838c1ec84807cdc913ad70aab651e259e537ef923bb27fa" exitCode=0
Sep 05 00:10:01.975378 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.975221    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"
Sep 05 00:10:01.975378 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.975267    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx" event={"ID":"57a30b88-b353-4314-8445-e8f7ed7635c2","Type":"ContainerDied","Data":"f417afe0ee0c86127838c1ec84807cdc913ad70aab651e259e537ef923bb27fa"}
Sep 05 00:10:01.975378 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.975318    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx" event={"ID":"57a30b88-b353-4314-8445-e8f7ed7635c2","Type":"ContainerDied","Data":"04427d3bd441074ead5595d4846eee5b020b2aa44d29eb24e45b50081880fa07"}
Sep 05 00:10:01.975378 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.975342    2564 scope.go:117] "RemoveContainer" containerID="f417afe0ee0c86127838c1ec84807cdc913ad70aab651e259e537ef923bb27fa"
Sep 05 00:10:01.986089 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.986072    2564 scope.go:117] "RemoveContainer" containerID="f417afe0ee0c86127838c1ec84807cdc913ad70aab651e259e537ef923bb27fa"
Sep 05 00:10:01.986355 ip-10-0-139-127 kubenswrapper[2564]: E0905 00:10:01.986331    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f417afe0ee0c86127838c1ec84807cdc913ad70aab651e259e537ef923bb27fa\": container with ID starting with f417afe0ee0c86127838c1ec84807cdc913ad70aab651e259e537ef923bb27fa not found: ID does not exist" containerID="f417afe0ee0c86127838c1ec84807cdc913ad70aab651e259e537ef923bb27fa"
Sep 05 00:10:01.986454 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.986361    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f417afe0ee0c86127838c1ec84807cdc913ad70aab651e259e537ef923bb27fa"} err="failed to get container status \"f417afe0ee0c86127838c1ec84807cdc913ad70aab651e259e537ef923bb27fa\": rpc error: code = NotFound desc = could not find container \"f417afe0ee0c86127838c1ec84807cdc913ad70aab651e259e537ef923bb27fa\": container with ID starting with f417afe0ee0c86127838c1ec84807cdc913ad70aab651e259e537ef923bb27fa not found: ID does not exist"
Sep 05 00:10:01.998224 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.998178    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"]
Sep 05 00:10:02.000005 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:01.999985    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-66b67c778f-rx9hx"]
Sep 05 00:10:02.174094 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:02.174059    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57a30b88-b353-4314-8445-e8f7ed7635c2" path="/var/lib/kubelet/pods/57a30b88-b353-4314-8445-e8f7ed7635c2/volumes"
Sep 05 00:10:18.751276 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:18.751237    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69b4b9b8d-sdgv5_2283e3ad-02f4-470e-9e4d-b8cfd4912968/authorino/0.log"
Sep 05 00:10:24.219795 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:24.219757    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qx95c_11cb0f47-96ed-49e5-9607-9934a2a3dabb/manager/0.log"
Sep 05 00:10:24.831195 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:24.831166    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b9496bc86-krnwn_6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6/copy-manifests/0.log"
Sep 05 00:10:24.837169 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:24.837149    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b9496bc86-krnwn_6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6/manager/0.log"
Sep 05 00:10:25.398655 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:25.398623    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-57989d57b9-shp75_df1bfa98-c2d8-4408-8458-9a2c106dc02c/maas-api/0.log"
Sep 05 00:10:25.494040 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:25.494013    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29809440-cxv58_80df0ae5-caec-4846-963e-053de587bb78/cleanup/0.log"
Sep 05 00:10:26.352094 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:26.352058    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69b4b9b8d-sdgv5_2283e3ad-02f4-470e-9e4d-b8cfd4912968/authorino/0.log"
Sep 05 00:10:26.816545 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:26.816495    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-28f8c_8cafd31f-a78a-4218-b193-78da58d65756/manager/0.log"
Sep 05 00:10:27.737290 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:27.737257    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d989d9b5-nppzf_71b6a21b-ccbc-465d-80e0-b9d235f75a3d/kube-auth-proxy/0.log"
Sep 05 00:10:27.966826 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:27.966785    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-4r6np_e3ba54fc-16ee-4eee-9826-5cd838046f10/payload-pre-processing/0.log"
Sep 05 00:10:28.632422 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:28.632390    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-gfqvz_c9bab180-6f5b-454e-9662-0c45bdd82baf/main/0.log"
Sep 05 00:10:28.864433 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:28.864405    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-csgh4_d911a1b7-e4b8-4984-9521-30310391bc95/main/0.log"
Sep 05 00:10:28.974707 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:28.974624    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbptxf_8cd22c48-6a0a-442e-bb85-c05e0a4881d8/main/0.log"
Sep 05 00:10:34.536639 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:34.536608    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:10:34.589922 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:34.589870    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log" limit=1024
Sep 05 00:10:37.714395 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:37.714365    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-krlhk_8678835d-111a-4d74-84e3-372e11e75a50/global-pull-secret-syncer/0.log"
Sep 05 00:10:37.840637 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:37.840609    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-29jcq_60746859-3df1-405d-88e6-4c686ac70181/konnectivity-agent/0.log"
Sep 05 00:10:37.996953 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:37.996875    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-127.ec2.internal_50d0ca4f5b95b0a1971e1cd22b27665d/haproxy/0.log"
Sep 05 00:10:42.741701 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:42.741671    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69b4b9b8d-sdgv5_2283e3ad-02f4-470e-9e4d-b8cfd4912968/authorino/0.log"
Sep 05 00:10:42.871954 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:42.871922    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-28f8c_8cafd31f-a78a-4218-b193-78da58d65756/manager/0.log"
Sep 05 00:10:44.322771 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:44.322734    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d98184a6-55e8-4aaa-a060-5128eec037d9/alertmanager/0.log"
Sep 05 00:10:44.343296 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:44.343259    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d98184a6-55e8-4aaa-a060-5128eec037d9/config-reloader/0.log"
Sep 05 00:10:44.365703 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:44.365685    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d98184a6-55e8-4aaa-a060-5128eec037d9/kube-rbac-proxy-web/0.log"
Sep 05 00:10:44.390641 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:44.390624    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d98184a6-55e8-4aaa-a060-5128eec037d9/kube-rbac-proxy/0.log"
Sep 05 00:10:44.411516 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:44.411475    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d98184a6-55e8-4aaa-a060-5128eec037d9/kube-rbac-proxy-metric/0.log"
Sep 05 00:10:44.446130 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:44.446111    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d98184a6-55e8-4aaa-a060-5128eec037d9/prom-label-proxy/0.log"
Sep 05 00:10:44.466108 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:44.466091    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d98184a6-55e8-4aaa-a060-5128eec037d9/init-config-reloader/0.log"
Sep 05 00:10:44.741851 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:44.741821    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p658s_ec473b55-9e99-4b68-ae23-b7eae54dc361/node-exporter/0.log"
Sep 05 00:10:44.763738 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:44.763711    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p658s_ec473b55-9e99-4b68-ae23-b7eae54dc361/kube-rbac-proxy/0.log"
Sep 05 00:10:44.783411 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:44.783392    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p658s_ec473b55-9e99-4b68-ae23-b7eae54dc361/init-textfile/0.log"
Sep 05 00:10:44.991794 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:44.991717    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9511166b-ed6c-40fa-916e-38dcf55e91e2/prometheus/0.log"
Sep 05 00:10:45.011566 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:45.011542    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9511166b-ed6c-40fa-916e-38dcf55e91e2/config-reloader/0.log"
Sep 05 00:10:45.035550 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:45.035528    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9511166b-ed6c-40fa-916e-38dcf55e91e2/thanos-sidecar/0.log"
Sep 05 00:10:45.055375 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:45.055355    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9511166b-ed6c-40fa-916e-38dcf55e91e2/kube-rbac-proxy-web/0.log"
Sep 05 00:10:45.081667 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:45.081646    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9511166b-ed6c-40fa-916e-38dcf55e91e2/kube-rbac-proxy/0.log"
Sep 05 00:10:45.110557 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:45.110539    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9511166b-ed6c-40fa-916e-38dcf55e91e2/kube-rbac-proxy-thanos/0.log"
Sep 05 00:10:45.133784 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:45.133766    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9511166b-ed6c-40fa-916e-38dcf55e91e2/init-config-reloader/0.log"
Sep 05 00:10:45.164840 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:45.164814    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-t4t9r_3bb52f2e-0f57-400d-b35b-88264721d4f6/prometheus-operator/0.log"
Sep 05 00:10:45.192541 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:45.192522    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-t4t9r_3bb52f2e-0f57-400d-b35b-88264721d4f6/kube-rbac-proxy/0.log"
Sep 05 00:10:45.225384 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:45.225360    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-qmp45_92ec9bce-ddb1-4d7e-91c8-c963bbaebb41/prometheus-operator-admission-webhook/0.log"
Sep 05 00:10:45.267715 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:45.267646    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5984c4d6f6-4zg6f_4d676e9d-9fb7-4cab-aba5-86f51c28ace4/telemeter-client/0.log"
Sep 05 00:10:45.288866 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:45.288842    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5984c4d6f6-4zg6f_4d676e9d-9fb7-4cab-aba5-86f51c28ace4/reload/0.log"
Sep 05 00:10:45.309708 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:45.309686    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5984c4d6f6-4zg6f_4d676e9d-9fb7-4cab-aba5-86f51c28ace4/kube-rbac-proxy/0.log"
Sep 05 00:10:47.541094 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.541061    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"]
Sep 05 00:10:47.541541 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.541417    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57a30b88-b353-4314-8445-e8f7ed7635c2" containerName="maas-api"
Sep 05 00:10:47.541541 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.541427    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a30b88-b353-4314-8445-e8f7ed7635c2" containerName="maas-api"
Sep 05 00:10:47.541541 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.541480    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="57a30b88-b353-4314-8445-e8f7ed7635c2" containerName="maas-api"
Sep 05 00:10:47.544698 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.544678    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.547256 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.547211    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c2q7w\"/\"default-dockercfg-8v5v9\""
Sep 05 00:10:47.548445 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.548425    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c2q7w\"/\"openshift-service-ca.crt\""
Sep 05 00:10:47.548552 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.548458    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c2q7w\"/\"kube-root-ca.crt\""
Sep 05 00:10:47.548710 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.548694    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9f2c3056-5935-4065-aaaf-91b98ccc5692-podres\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.548772 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.548744    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f2c3056-5935-4065-aaaf-91b98ccc5692-lib-modules\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.548825 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.548763    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f2c3056-5935-4065-aaaf-91b98ccc5692-sys\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.548825 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.548819    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzhqz\" (UniqueName: \"kubernetes.io/projected/9f2c3056-5935-4065-aaaf-91b98ccc5692-kube-api-access-xzhqz\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.548940 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.548921    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9f2c3056-5935-4065-aaaf-91b98ccc5692-proc\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.552563 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.552541    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"]
Sep 05 00:10:47.649868 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.649830    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9f2c3056-5935-4065-aaaf-91b98ccc5692-podres\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.650031 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.649937    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f2c3056-5935-4065-aaaf-91b98ccc5692-lib-modules\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.650031 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.649963    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f2c3056-5935-4065-aaaf-91b98ccc5692-sys\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.650031 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.649987    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzhqz\" (UniqueName: \"kubernetes.io/projected/9f2c3056-5935-4065-aaaf-91b98ccc5692-kube-api-access-xzhqz\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.650031 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.650008    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9f2c3056-5935-4065-aaaf-91b98ccc5692-podres\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.650250 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.650068    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9f2c3056-5935-4065-aaaf-91b98ccc5692-proc\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.650250 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.650073    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f2c3056-5935-4065-aaaf-91b98ccc5692-sys\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.650250 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.650097    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f2c3056-5935-4065-aaaf-91b98ccc5692-lib-modules\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.650250 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.650128    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9f2c3056-5935-4065-aaaf-91b98ccc5692-proc\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.657554 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.657527    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzhqz\" (UniqueName: \"kubernetes.io/projected/9f2c3056-5935-4065-aaaf-91b98ccc5692-kube-api-access-xzhqz\") pod \"perf-node-gather-daemonset-9v7hf\" (UID: \"9f2c3056-5935-4065-aaaf-91b98ccc5692\") " pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.855520 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.855469    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:47.860621 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:47.860603    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-86d4c4dc55-z4mcm_ba87952b-527d-4f5a-959c-dd1667ee7188/console/0.log"
Sep 05 00:10:48.125003 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:48.124972    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"]
Sep 05 00:10:48.127149 ip-10-0-139-127 kubenswrapper[2564]: W0905 00:10:48.127123    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9f2c3056_5935_4065_aaaf_91b98ccc5692.slice/crio-eda28877e0f762cfb90eb30120160f3a48788c862caf9a7605f1eb7338b3dcb7 WatchSource:0}: Error finding container eda28877e0f762cfb90eb30120160f3a48788c862caf9a7605f1eb7338b3dcb7: Status 404 returned error can't find the container with id eda28877e0f762cfb90eb30120160f3a48788c862caf9a7605f1eb7338b3dcb7
Sep 05 00:10:48.193201 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:48.193173    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf" event={"ID":"9f2c3056-5935-4065-aaaf-91b98ccc5692","Type":"ContainerStarted","Data":"261b09e3e89e6e53842669ec029e44ef9a783e72ea29e8f8abd1ceee28389364"}
Sep 05 00:10:48.193297 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:48.193220    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf" event={"ID":"9f2c3056-5935-4065-aaaf-91b98ccc5692","Type":"ContainerStarted","Data":"eda28877e0f762cfb90eb30120160f3a48788c862caf9a7605f1eb7338b3dcb7"}
Sep 05 00:10:49.197859 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:49.197827    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:49.214010 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:49.213963    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf" podStartSLOduration=2.213947759 podStartE2EDuration="2.213947759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 00:10:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 00:10:49.211827018 +0000 UTC m=+2037.459827547" watchObservedRunningTime="2026-09-05 00:10:49.213947759 +0000 UTC m=+2037.461948275"
Sep 05 00:10:49.653952 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:49.653921    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dblvw_d08e3b44-f805-4369-becc-e73602a975c7/dns/0.log"
Sep 05 00:10:49.673023 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:49.672998    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dblvw_d08e3b44-f805-4369-becc-e73602a975c7/kube-rbac-proxy/0.log"
Sep 05 00:10:49.735678 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:49.735648    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2q7dw_57fd64bb-51ba-4701-81bc-fbff66028fb2/dns-node-resolver/0.log"
Sep 05 00:10:50.351951 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:50.351918    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4l98m_2b94086d-38f4-4be0-857d-ae657478079d/node-ca/0.log"
Sep 05 00:10:51.897269 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:51.897240    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d989d9b5-nppzf_71b6a21b-ccbc-465d-80e0-b9d235f75a3d/kube-auth-proxy/0.log"
Sep 05 00:10:51.975648 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:51.975618    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-4r6np_e3ba54fc-16ee-4eee-9826-5cd838046f10/payload-pre-processing/0.log"
Sep 05 00:10:52.907752 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:52.907729    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-np8fh_f4f200b7-9fb1-457f-9e3a-f85901981765/serve-healthcheck-canary/0.log"
Sep 05 00:10:53.508642 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:53.508610    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x7wjj_222b3a9b-c948-4aad-88e1-4dde66f6f993/kube-rbac-proxy/0.log"
Sep 05 00:10:53.527076 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:53.527048    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x7wjj_222b3a9b-c948-4aad-88e1-4dde66f6f993/exporter/0.log"
Sep 05 00:10:53.546258 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:53.546232    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x7wjj_222b3a9b-c948-4aad-88e1-4dde66f6f993/extractor/0.log"
Sep 05 00:10:55.211866 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:55.211841    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c2q7w/perf-node-gather-daemonset-9v7hf"
Sep 05 00:10:55.724245 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:55.724211    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qx95c_11cb0f47-96ed-49e5-9607-9934a2a3dabb/manager/0.log"
Sep 05 00:10:56.026954 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:56.026861    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b9496bc86-krnwn_6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6/manager/0.log"
Sep 05 00:10:56.047961 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:56.047936    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b9496bc86-krnwn_6ad6cdca-685c-4ec5-acbf-b1d9a86c2ed6/copy-manifests/0.log"
Sep 05 00:10:57.610659 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:57.610623    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58b866555c-g6ls7_10d1c54a-cee6-4fa3-8c35-a6a022a8b315/manager/0.log"
Sep 05 00:10:58.886891 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:58.886864    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d989d9b5-nppzf_71b6a21b-ccbc-465d-80e0-b9d235f75a3d/kube-auth-proxy/0.log"
Sep 05 00:10:58.957756 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:10:58.957734    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-4r6np_e3ba54fc-16ee-4eee-9826-5cd838046f10/payload-pre-processing/0.log"
Sep 05 00:11:01.021672 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:11:01.021641    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d989d9b5-nppzf_71b6a21b-ccbc-465d-80e0-b9d235f75a3d/kube-auth-proxy/0.log"
Sep 05 00:11:01.088253 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:11:01.088230    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-4r6np_e3ba54fc-16ee-4eee-9826-5cd838046f10/payload-pre-processing/0.log"
Sep 05 00:11:04.266535 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:11:04.266502    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8z2h_f683ee03-698b-4e74-8fbb-427327ec0ff3/kube-multus-additional-cni-plugins/0.log"
Sep 05 00:11:04.286865 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:11:04.286840    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8z2h_f683ee03-698b-4e74-8fbb-427327ec0ff3/egress-router-binary-copy/0.log"
Sep 05 00:11:04.306978 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:11:04.306957    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8z2h_f683ee03-698b-4e74-8fbb-427327ec0ff3/cni-plugins/0.log"
Sep 05 00:11:04.326960 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:11:04.326939    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8z2h_f683ee03-698b-4e74-8fbb-427327ec0ff3/bond-cni-plugin/0.log"
Sep 05 00:11:04.347407 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:11:04.347387    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8z2h_f683ee03-698b-4e74-8fbb-427327ec0ff3/routeoverride-cni/0.log"
Sep 05 00:11:04.367181 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:11:04.367159    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8z2h_f683ee03-698b-4e74-8fbb-427327ec0ff3/whereabouts-cni-bincopy/0.log"
Sep 05 00:11:04.386057 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:11:04.386036    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-b8z2h_f683ee03-698b-4e74-8fbb-427327ec0ff3/whereabouts-cni/0.log"
Sep 05 00:11:04.576554 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:11:04.576488    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ff5c2_4b6d3fb7-41a9-4ebd-aa16-7d70165952bb/kube-multus/0.log"
Sep 05 00:11:04.712805 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:11:04.712777    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fk66m_202d6558-e746-4911-9b99-f1b079343126/network-metrics-daemon/0.log"
Sep 05 00:11:04.730306 ip-10-0-139-127 kubenswrapper[2564]: I0905 00:11:04.730278    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fk66m_202d6558-e746-4911-9b99-f1b079343126/kube-rbac-proxy/0.log"